Mamba_State_Space_Model_Paper_List

Paper list for State-Space-Model and its Applications

We appreciate any useful suggestions for improvement of this paper list or survey from peers. Please raise issues or send an email to [email protected]. Thanks for your cooperation!

State Space Model for New-Generation Network Alternative to Transformers: A Survey, Xiao Wang, Shiao Wang, Yuhe Ding, Yuehang Li, Wentao Wu, Yao Rong, Weizhe Kong, Ju Huang, Shihao Li, Haoxiang Yang, Ziwen Wang, Bo Jiang, Chenglong Li, Yaowei Wang, Yonghong Tian, Jin Tang, 2024 [PDF] [arXiv]

:collision: Update Log

• [2024.04.15] We release the first version of the survey on state space model [arXiv]

Thesis & Surveys

• Modeling sequences with structured state spaces, Responsibility: Albert Gu, Publication: [Stanford, California] : [Stanford University], 2023 [Thesis (330 pages)] [PDF]

• State Space Model for New-Generation Network Alternative to Transformers: A Survey, Xiao Wang, Shiao Wang, Yuhe Ding, Yuehang Li, Wentao Wu, Yao Rong, Weizhe Kong, Ju Huang, Shihao Li, Haoxiang Yang, Ziwen Wang, Bo Jiang, Chenglong Li, Yaowei Wang, Yonghong Tian, Jin Tang, 2024 [PDF] [arXiv]

• State Space Models as Foundation Models: A Control Theoretic Overview, arXiv:2403.16899, Carmen Amo Alonso, Jerome Sieber, Melanie N. Zeilinger [Paper]

• A Survey on Visual Mamba, arXiv:2404.15956, Hanwei Zhang, Ying Zhu, Dan Wang, Lijun Zhang, Tianxiang Chen, Zi Ye [Paper]

• Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges, Badri Narayana Patro, Vijay Srinivas Agneeswaran [Paper] [Github]

• A Survey on Vision Mamba: Models, Applications and Challenges, Rui Xu, Shu Yang, Yihui Wang, Bo Du, Hao Chen [Paper] [Paper List]

Year 2024

• CLIP-Mamba: CLIP Pretrained Mamba Models with OOD and Hessian Evaluation, arXiv:2404.19394, Weiquan Huang, Yifei Shen, Yifan Yang [Paper] [Code]

• Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models, arXiv:2404.18508, Mark Schöne, Neeraj Mohan Sushma, Jingyue Zhuge, Christian Mayr, Anand Subramoney, David Kappel [Paper]

• Revisiting Multi-modal Emotion Learning with Broad State Space Models and Probability-guidance Fusion, arXiv:2404.17858, Yuntao Shou, Tao Meng, Fuchen Zhang, Nan Yin, Keqin Li [Paper]

• Spectral-Spatial Mamba for Hyperspectral Image Classification, arXiv:2404.18401, Lingbo Huang, Yushi Chen, Xin He [Paper]

• RSCaMa: Remote Sensing Image Change Captioning with State Space Model, Chenyang Liu, Keyan Chen, Bowen Chen, Haotian Zhang, Zhengxia Zou, Zhenwei Shi [Paper] [Code]

• S2Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification, Guanchun Wang, Xiangrong Zhang, Zelin Peng, Tianyang Zhang, Xiuping Jia, Licheng Jiao [Paper] [Code]

• Mamba-FETrack: Frame-Event Tracking via State Space Model, arXiv:2404.18174, Ju Huang, Shiao Wang, Shuai Wang, Zhe Wu, Xiao Wang, Bo Jiang [Paper] [Code]

• Bi-Mamba4TS: Bidirectional Mamba for Time Series Forecasting, arXiv:2404.15772, Aobo Liang, Xingguo Jiang, Yan Sun, Chang Lu [Paper]

• ST-MambaSync: The Confluence of Mamba Structure and Spatio-Temporal Transformers for Precipitous Traffic Prediction, arXiv:2404.15899 Zhiqi Shao, Xusheng Yao, Ze Wang, Junbin Gao [Paper]

• Optimizing Universal Lesion Segmentation: State Space Model-Guided Hierarchical Networks with Feature Importance Adjustment, Kazi Shahriar Sanjid, Md. Tanzim Hossain, Md. Shakib Shahariar Junayed, M. Monir Uddin [Paper]

• Sparse Reconstruction of Optical Doppler Tomography Based on State Space Model, arXiv:2404.17484, Zhenghong Li, Jiaxiang Ren, Wensheng Cheng, Congwu Du, Yingtian Pan, Haibin Ling [Paper]

• CFMW: Cross-modality Fusion Mamba for Multispectral Object Detection under Adverse Weather Conditions, arXiv:2404.16302, Haoyuan Li, Qi Hu, You Yao, Kailun Yang, Peng Chen [Paper] [Code]

• MambaTrack: Exploring Learning-based Motion Models in Multi-Object Tracking, arXiv:2403.10826, Hsiang-Wei Huang, Cheng-Yen Yang, Wenhao Chai, Zhongyu Jiang, Jenq-Neng Hwang [Paper]

• MambaUIE&SR: Unraveling the Ocean's Secrets with Only 2.8 FLOPs, arXiv:2404.13884, Zhihao Chen, Yiyuan Ge [Paper] [Code]

• Vim4Path: Self-Supervised Vision Mamba for Histopathology Images, arXiv:2404.13222, Ali Nasiri-Sarvi, Vincent Quoc-Huy Trinh, Hassan Rivaz, Mahdi S. Hosseini [Paper] [Code]

• Integrating Mamba and Transformer for Long-Short Range Time Series Forecasting, arXiv:2404.14757, Xiongxiao Xu, Yueqing Liang, Baixiang Huang, Zhiling Lan, Kai Shu [Paper] [Code]

• Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model, arXiv:2404.14966, Xu Han, Yuan Tang, Zhaoxuan Wang, Xianzhi Li [Paper]

• ST-SSMs: Spatial-Temporal Selective State of Space Model for Traffic Forecasting, arXiv:2404.13257, Zhiqi Shao, Michael G.H. Bell, Ze Wang, D. Glenn Geers, Haoning Xi, Junbin Gao [Paper]

• The Illusion of State in State-Space Models, arXiv:2404.08819 William Merrill, Jackson Petty, Ashish Sabharwal [Paper]

• MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model, Kang Zeng, Hao Shi, Jiacheng Lin, Siyu Li, Jintao Cheng, Kaiwei Wang, Zhiyong Li, Kailun Yang [Paper] [Code]

• MambaPupil: Bidirectional Selective Recurrent model for Event-based Eye tracking, CVPR 2024 Workshop (AIS: Vision, Graphics and AI for Streaming) Zhong Wang, Zengyu Wan, Han Han, Bohao Liao, Yuliang Wu, Wei Zhai, Yang Cao, Zheng-jun Zha [Paper]

• CU-Mamba: Selective State Space Models with Channel Learning for Image Restoration, arXiv:2404.11778, Rui Deng, Tianpei Gu [Paper]

• State-space Decomposition Model for Video Prediction Considering Long-term Motion Trend, Fei Cui, Jiaojiao Fang, Xiaojiang Wu, Zelong Lai, Mengke Yang, Menghan Jia, Guizhong Liu [Paper]

• Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion, Xinghan Wang, Zixi Kang, Yadong Mu, arXiv:2404.11375 [Paper]

• HumMUSS: Human Motion Understanding using State Space Models, Arnab Kumar Mondal, Stefano Alletto, Denis Tome, CVPR 2024, arXiv:2404.10880 [Paper]

• HSIDMamba: Exploring Bidirectional State-Space Models for Hyperspectral Denoising, Yang Liu, Jiahua Xiao, Yu Guo, Peilin Jiang, Haiwei Yang, Fei Wang [Paper]

• FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba, Xinyu Xie, Yawen Cui, Chio-In Ieong, Tao Tan, Xiaozhi Zhang, Xubin Zheng, Zitong Yu [Paper] [Code]

• FreqMamba: Viewing Mamba from a Frequency Perspective for Image Deraining, Zou Zhen, Yu Hu, Zhao Feng [Paper]

• A Novel State Space Model with Local Enhancement and State Sharing for Image Fusion, Zihan Cao, Xiao Wu, Liang-Jian Deng, Yu Zhong [Paper]

• Fusion-Mamba for Cross-modality Object Detection, arXiv:2404.09146, Wenhao Dong, Haodong Zhu, Shaohui Lin, Xiaoyan Luo, Yunhang Shen, Xuhui Liu, Juan Zhang, Guodong Guo, Baochang Zhang [Paper]

• "Linear recurrent units for sequential recommendation." Yue, Zhenrui, et al. Proceedings of the 17th ACM International Conference on Web Search and Data Mining. 2024. [Paper] [Code]

• State-Space Modeling of Shape-constrained Functional Time Series, Daichi Hiraki, Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa, arXiv:2404.07586 [Paper]

• HGRN2: Gated Linear RNNs with State Expansion, Zhen Qin, Songlin Yang, Weixuan Sun, Xuyang Shen, Dong Li, Weigao Sun, Yiran Zhong, arXiv:2404.07904 [Paper] [Code]

• MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion, Zhe Li, Haiwei Pan, Kejia Zhang, Yuhua Wang, Fengming Yu, arXiv:2404.08406 [Paper]

• SpectralMamba: Efficient Mamba for Hyperspectral Image Classification, Jing Yao, Danfeng Hong, Chenyu Li, Jocelyn Chanussot, arXiv:2404.08489 [Paper] [Code]

• SurvMamba: State Space Model with Multi-grained Multi-modal Interaction for Survival Prediction, Ying Chen, Jiajing Xie, Yuxiang Lin, Yuhang Song, Wenxian Yang, Rongshan Yu, arXiv:2404.08027 [Paper]

• [2024_143] FusionMamba: Efficient Image Fusion with State Space Model, Siran Peng, Xiangyu Zhu, Haoyu Deng, Zhen Lei, Liang-Jian Deng [Paper]

• [2024_142] DGMamba: Domain Generalization via Generalized State Space Model, Shaocong Long, Qianyu Zhou, Xiangtai Li, Xuequan Lu, Chenhao Ying, Yuan Luo, Lizhuang Ma, Shuicheng Yan [Paper]

• [2024_141] ViM-UNet: Vision Mamba for Biomedical Segmentation, Anwai Archit, Constantin Pape [Paper] [Code]

• [2024_140] Simba: Mamba augmented U-ShiftGCN for Skeletal Action Recognition in Videos, Soumyabrata Chaudhuri, Saumik Bhattacharya [Paper]

• [2024_139] MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection, Haoyang He, Yuhu Bai, Jiangning Zhang, Qingdong He, Hongxu Chen, Zhenye Gan, Chengjie Wang, Xiangtai Li, Guanzhong Tian, Lei Xie [Paper]

• [2024_138] 3DMambaComplete: Exploring Structured State Space Model for Point Cloud Completion, Yixuan Li, Weidong Yang, Ben Fei [Paper]

• [2024_137] RhythmMamba: Fast Remote Physiological Measurement with Arbitrary Length Videos, Bochao Zou, Zizheng Guo, Xiaocheng Hu, Huimin Ma [Paper] [Code]

• [2024_136] VMambaMorph: a Visual Mamba-based Framework with Cross-Scan Module for Deformable 3D Image Registration, Ziyang Wang, Jian-Qing Zheng, Chao Ma, Tao Guo [Paper]

• [2024_135] 3DMambaIPF: A State Space Model for Iterative Point Cloud Filtering via Differentiable Rendering, Qingyuan Zhou, Weidong Yang, Ben Fei, Jingyi Xu, Rui Zhang, Keyi Liu, Yeqi Luo, Ying He [Paper]

• [2024_134] Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation, Zifu Wan, Yuhao Wang, Silong Yong, Pingping Zhang, Simon Stepputtis, Katia Sycara, Yaqi Xie [Paper] [Code]

• [2024_133] xT: Nested Tokenization for Larger Context in Large Images, Ritwik Gupta, Shufan Li, Tyler Zhu, Jitendra Malik, Trevor Darrell, Karttikeya Mangalam [Paper] [Code]

• [2024_132] Locating and Editing Factual Associations in Mamba, Arnab Sen Sharma, David Atkinson, David Bau [Paper]

• [2024_131] InsectMamba: Insect Pest Classification with State Space Model, Qianning Wang, Chenglin Wang, Zhixin Lai, Yucheng Zhou [Paper]

• [2024_130] ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model, Hongruixuan Chen, Jian Song, Chengxi Han, Junshi Xia, Naoto Yokoya [Paper] [Code]

• [2024_129] RS-Mamba for Large Remote Sensing Image Dense Prediction, Sijie Zhao, Hao Chen, Xueliang Zhang, Pengfeng Xiao, Lei Bai, Wanli Ouyang [Paper] [Code]

• [2024_128] RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation, Xianping Ma, Xiaokang Zhang, Man-On Pun [Paper] [Code]

• [2024_127] SPMamba: State-space model is all you need in speech separation, Kai Li, Guo Chen [Paper]

• [2024_126] On the reduction of Linear Parameter-Varying State-Space models, E. Javier Olucha, Bogoljub Terzin, Amritam Das, Roland Tóth [Paper]

• [2024_125] Samba: Semantic Segmentation of Remotely Sensed Images with State Space Model, Qinfeng Zhu, Yuanzhi Cai, Yuan Fang, Yihan Yang, Cheng Chen, Lei Fan, Anh Nguyen [Paper] [Code]

• [2024_124] T-Mamba: Frequency-Enhanced Gated Long-Range Dependency for Tooth 3D CBCT Segmentation, Jing Hao, Lei He, Kuo Feng Hung [Paper] [Code]

• [2024_123] Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces, Toshihiro Ota [Paper]

• [2024_122] RankMamba, Benchmarking Mamba's Document Ranking Performance in the Era of Transformers, Zhichao Xu [Paper] [Code]

• [2024_121] SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding, Wenrui Li, Xiaopeng Hong, Xiaopeng Fan [Paper]

• [2024_120] HSIMamba: Hyperpsectral Imaging Efficient Feature Learning with Bidirectional State Space for Classification, Judy X Yang, Jun Zhou, Jing Wang, Hui Tian, Alan Wee Chung Liew [Paper] [Code]

• [2024_119] HARMamba: Efficient Wearable Sensor Human Activity Recognition Based on Bidirectional Selective SSM, Shuangjian Li, Tao Zhu, Furong Duan, Liming Chen, Huansheng Ning, Yaping Wan [Paper]

• [2024_118] UltraLight VM-UNet: Parallel Vision Mamba Significantly Reduces Parameters for Skin Lesion Segmentation, Renkai Wu, Yinghao Liu, Pengchen Liang, Qing Chang [Paper] [Code]

• [2024_117] MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection, Ali Behrouz, Michele Santacatterina, Ramin Zabih [Paper]

• [2024_116] Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation, Xilin Jiang, Cong Han, Nima Mesgarani [Paper]

• [2024_115] STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model, Lincan Li, Hanchen Wang, Wenjie Zhang, Adelle Coster [Paper]

• [2024_114] Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference, Han Zhao, Min Zhang, Wei Zhao, Pengxiang Ding, Siteng Huang, Donglin Wang [Paper]

• [2024_113] Music to Dance as Language Translation using Sequence Models, André Correia, Luís A. Alexandre [Paper] [Code]

• [2024_112] CMViM: Contrastive Masked Vim Autoencoder for 3D Multi-modal Representation Learning for AD classification, Guangqian Yang, Kangrui Du, Zhihan Yang, Ye Du, Yongping Zheng, Shujun Wang [Paper]

• [2024_111] Proprioception Is All You Need: Terrain Classification for Boreal Forests, Damien LaRocque, William Guimont-Martin, David-Alexandre Duclos, Philippe Giguère, François Pomerleau [Paper]

• [2024_110] ReMamber: Referring Image Segmentation with Mamba Twister, Yuhuan Yang, Chaofan Ma, Jiangchao Yao, Zhun Zhong, Ya Zhang, Yanfeng Wang [Paper]

• [2024_109] Mechanistic Design and Scaling of Hybrid Architectures, Michael Poli, Armin W Thomas, Eric Nguyen, Pragaash Ponnusamy, Björn Deiseroth, Kristian Kersting, Taiji Suzuki, Brian Hie, Stefano Ermon, Christopher Ré, Ce Zhang, Stefano Massaroli [Paper]

• [2024_108] Model order reduction of deep structured state-space models: A system-theoretic approach, Marco Forgione, Manas Mejari, Dario Piga

• [2024_107] Modeling Analog Dynamic Range Compressors using Deep Learning and State-space Models, Hanzhi Yin, Gang Cheng, Christian J. Steinmetz, Ruibin Yuan, Richard M. Stern, Roger B. Dannenberg [Paper]

• [2024_106] Uncovering Selective State Space Model's Capabilities in Lifelong Sequential Recommendation, Jiyuan Yang, Yuanzi Li, Jingyu Zhao, Hanbing Wang, Muyang Ma, Jun Ma, Zhaochun Ren, Mengqi Zhang, Xin Xin, Zhumin Chen, Pengjie Ren [Paper] [Code]

• [2024_105]

• [2024_104] Serpent: Scalable and Efficient Image Restoration via Multi-scale Structured State Space Models, Mohammad Shahab Sepehri, Zalan Fabian, Mahdi Soltanolkotabi [Paper]

• [2024_103] Jamba: A Hybrid Transformer-Mamba Language Model, Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, Omri Abend, Raz Alon, Tomer Asida, Amir Bergman, Roman Glozman, Michael Gokhman, Avashalom Manevich, Nir Ratner, Noam Rozen, Erez Shwartz, Mor Zusman, Yoav Shoham [Paper] [Website] [Huggingface]

• [2024_102] Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction, Qiuhong Shen, Xuanyu Yi, Zike Wu, Pan Zhou, Hanwang Zhang, Shuicheng Yan, Xinchao Wang [Paper]

• [2024_101] RSMamba: Remote Sensing Image Classification with State Space Model, [Project] [Paper] [Code]

• [2024_100] Rotate to Scan: UNet-like Mamba with Triplet SSM Module for Medical Image Segmentation, Hao Tang, Lianglun Cheng, Guoheng Huang, Zhengguang Tan, Junhao Lu, Kaihong Wu [Paper]

• [2024_099] PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition, Chenhongyi Yang, Zehui Chen, Miguel Espinosa, Linus Ericsson, Zhenyu Wang, Jiaming Liu, Elliot J. Crowley [Paper] [Code]

• [2024_098] Integrating Mamba Sequence Model and Hierarchical Upsampling Network for Accurate Semantic Segmentation of Multiple Sclerosis Legion, Kazi Shahriar Sanjid, Md. Tanzim Hossain, Md. Shakib Shahariar Junayed, Dr. Mohammad Monir Uddin [Paper]

• [2024_097] VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting, Yujin Tang, Peijie Dong, Zhenheng Tang, Xiaowen Chu, Junwei Liang [Paper] [Code]

• [2024_096] SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series, Badri N. Patro, Vijay S. Agneeswaran [Paper] [Code]

• [2024_095] Repeat After Me: Transformers are Better than State Space Models at Copying, Samy Jelassi, David Brandfonbrener, Sham M. Kakade, Eran Malach [Paper]

• [2024_094]H-vmunet: High-order Vision Mamba UNet for Medical Image Segmentation, Renkai Wu, Yinghao Liu, Pengchen Liang, Qing Chang [Paper] [Code]

• [2024_093]VL-Mamba: Exploring State Space Models for Multimodal Learning, Yanyuan Qiao, Zheng Yu, Longteng Guo, Sihan Chen, Zijia Zhao, Mingzhen Sun, Qi Wu, Jing Liu [Paper] [Project] [Code]

• [2024_092]ProMamba: Prompt-Mamba for polyp segmentation, Jianhao Xie, Ruofan Liao, Ziang Zhang, Sida Yi, Yuesheng Zhu, Guibo Luo [Paper]

• [2024_091]ZigMa: Zigzag Mamba Diffusion Model, Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Fischer, Bjorn Ommer [Paper] [Code]

• [2024_090]On the low-shot transferability of [V]-Mamba, Diganta Misra, Jay Gala, Antonio Orvieto [Paper]

• [2024_089]Is Mamba Effective for Time Series Forecasting? Zihan Wang, Fanheng Kong, Shi Feng, Ming Wang, Han Zhao, Daling Wang, Yifei Zhang [Paper] [Code]

• [2024_088]VmambaIR: Visual State Space Model for Image Restoration, Yuan Shi, Bin Xia, Xiaoyu Jin, Xing Wang, Tianyu Zhao, Xin Xia, Xuefeng Xiao, Wenming Yang [Paper]

• [2024_087]Understanding Robustness of Visual State Space Models for Image Classification, Chengbin Du, Yanxi Li, Chang Xu [Paper]

• [2024_086]Regularization-Based Efficient Continual Learning in Deep State-Space Models, Yuanhang Zhang, Zhidi Lin, Yiyong Sun, Feng Yin, Carsten Fritsche [Paper]

• [2024_085]TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting, Md Atik Ahamed, Qiang Cheng [Paper] [Code]

• [2024_084]EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba, Xiaohuan Pei, Tao Huang, Chang Xu [Paper] [Code]

• [2024_083]MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models, Zunnan Xu, Yukang Lin, Haonan Han, Sicheng Yang, Ronghui Li, Yachao Zhang, Xiu Li [Paper]

• [2024_082]LocalMamba: Visual State Space Model with Windowed Selective Scan, Tao Huang, Xiaohuan Pei, Shan You, Fei Wang, Chen Qian, Chang Xu [Paper] [Code]

• [2024_081]VM-UNET-V2 Rethinking Vision Mamba UNet for Medical Image Segmentation, Mingya Zhang, Yue Yu, Limei Gu, Tingsheng Lin, Xianping Tao [Paper] [Code]

• [2024_080]Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding, Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, Limin Wang [Paper] [Code]

• [2024_079]Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling, Raunaq Bhirangi, Chenyu Wang, Venkatesh Pattabiraman, Carmel Majidi, Abhinav Gupta, Tess Hellebrekers, Lerrel Pinto [Paper]

• [2024_078]MambaStock: Selective state space model for stock prediction, Zhuangwei Shi [Paper] [Code]

• [2024_077]Simple linear attention language models balance the recall-throughput tradeoff, Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, Dylan Zinsley, James Zou, Atri Rudra, Christopher Ré [Paper]

• [2024_076]LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image Segmentation, Weibin Liao, Yinghao Zhu, Xinyuan Wang, Chengwei Pan, Yasha Wang, Liantao Ma [Paper] [Code]

• [2024_075]Motion-Guided Dual-Camera Tracker for Low-Cost Skill Evaluation of Gastric Endoscopy, Yuelin Zhang, Wanquan Yan, Kim Yan, Chun Ping Lam, Yufu Qiu, Pengyu Zheng, Raymond Shing-Yan Tang, Shing Shin Cheng [Paper] [Code]

• [2024_074]Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling, Yair Schiff, Chia-Hsiang Kao, Aaron Gokaslan, Tri Dao, Albert Gu, Volodymyr Kuleshov [Paper]

• [2024_073]MD-Dose: A Diffusion Model based on the Mamba for Radiotherapy Dose Prediction, Linjie Fu, Xia Li, Xiuding Cai, Yingkai Wang, Xueyao Wang, Yali Shen, Yu Yao [Paper]

• [2024_072]Activating Wider Areas in Image Super-Resolution, Cheng Cheng, Hang Wang, Hongbin Sun [Paper]

• [2024_071]Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakers, Changsheng Quan, Xiaofei Li [Paper]

• [2024_070]A multi-cohort study on prediction of acute brain dysfunction states using selective state space models, Brandon Silva, Miguel Contreras, Sabyasachi Bandyopadhyay, Yuanfang Ren, Ziyuan Guan, Jeremy Balch, Kia Khezeli, Tezcan Ozrazgat Baslanti, Ben Shickel, Azra Bihorac, Parisa Rashidi [Paper]

• [2024_069]The pitfalls of next-token prediction, Gregor Bachmann, Vaishnavh Nagarajan [Paper] [Code]

• [2024_068]Large Window-based Mamba UNet for Medical Image Segmentation: Beyond Convolution and Self-attention, Jinhong Wang, Jintai Chen, Danny Chen, Jian Wu [Paper] [Code]

• [2024_067]Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM, Zeyu Zhang, Akide Liu, Ian Reid, Richard Hartley, Bohan Zhuang, Hao Tang [Paper] [Project] [Code]

• [2024_066]ClinicalMamba: A Generative Clinical Language Model on Longitudinal Clinical Notes, Zhichao Yang, Avijit Mitra, Sunjae Kwon, Hong Yu [Paper] [Code]

• [2024_065]MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology, Shu Yang, Yihui Wang, Hao Chen [Paper] [Code]

• [2024_064]Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy, Jiuming Liu, Ruiji Yu, Yian Wang, Yu Zheng, Tianchen Deng, Weicai Ye, Hesheng Wang [Paper] [Code]

• [2024_063]VideoMamba: State Space Model for Efficient Video Understanding, Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Yali Wang, Limin Wang, Yu Qiao [Paper] [Code]

• [2024_062]MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models, Zijie Fang, Yifeng Wang, Zhi Wang, Jian Zhang, Xiangyang Ji, Yongbing Zhang [Paper]

• [2024_061]Video Diffusion State Space Models, Zhengcong Fei, Mingyuan Fan, Changqian Yu, Jusnshi Huang, [Paper] [Code]

• [2024_060]Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models, Chengkai Liu, Jianghao Lin, Jianling Wang, Hanzhou Liu, James Caverlee [Paper]

• [2024_059]MedMamba: Vision Mamba for Medical Image Classification, Yubiao Yue, Zhenzhang Li [Paper] [Code]

• [2024_058]Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models, Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando De Freitas, Caglar Gulcehre [Paper]

• [2024_057]Gated Linear Attention Transformers with Hardware-Efficient Training, Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, Yoon Kim [Paper] [Code]

• [2024_056]DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models, [Paper] [Code]

• [2024_055]The Hidden Attention of Mamba Models, [Paper] [Code]

• [2024_054]MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small Target Detection, Tianxiang Chen, Zhentao Tan, Tao Gong, Qi Chu, Yue Wu, Bin Liu, Jieping Ye, Nenghai Yu [Paper] [Code]

• [2024_053]Point Could Mamba: Point Cloud Learning via State Space Model, Tao Zhang, Xiangtai Li, Haobo Yuan, Shunping Ji, Shuicheng Yan [Paper] [Code]

• [2024_052]Res-VMamba: Fine-Grained Food Category Visual Classification Using Selective State Space Models with Deep Residual Learning, Chi-Sheng Chen, Guan-Ying Chen, Dong Zhou, Di Jiang, Dai-Shi Chen [Paper] [Code]

• [2024_051]MambaMIR: An Arbitrary-Masked Mamba for Joint Medical Image Reconstruction and Uncertainty Estimation, Jiahao Huang, Liutao Yang, Fanwen Wang, Yinzhe Wu, Yang Nan, Angelica I. Aviles-Rivero, Carola-Bibiane Schönlieb, Daoqiang Zhang, Guang Yang [Paper] [Code]

• [2024_050]MambaIR: A Simple Baseline for Image Restoration with State-Space Model, Hang Guo, Jinmin Li, Tao Dai, Zhihao Ouyang, Xudong Ren, Shu-Tao Xia [Paper] [Code]

• [2024_049]State Space Models for Event Cameras, Nikola Zubić, Mathias Gehrig, Davide Scaramuzza [Paper]

• [2024_048][ICLR 2024] FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores, Daniel Y Fu, Hermann Kumbong, Eric Nguyen, Christopher Re [Paper]

• [2024_047]Variational quantization for state space models, Etienne David, Jean Bellot, Sylvain Le Corff [Paper]

• [2024_046]Efficient Long Sequence Modeling via State Space Augmented Transformer, Simiao Zuo, Xiaodong Liu, Jian Jiao, Denis X Charles, Eren Manavoglu, Tuo Zhao, Jianfeng Gao [Paper]

• [2024_045][ICLR 2024] Robustifying State-space Models for Long Sequences via Approximate Diagonalization, Annan Yu, Arnur Nigmetov, Dmitriy Morozov, Michael W. Mahoney, N. Benjamin Erichson [Paper]

• [2024_044]From generalization analysis to optimization designs for state space models, Fusheng Liu, Qianxiao Li [Paper]

• [2024_043]A 2-Dimensional State Space Layer for Spatial Inductive Bias, Ethan Baron, Itamar Zimerman, Lior Wolf [Paper]

• [2024_042][ICLR 2024] Hieros: Hierarchical Imagination on Structured State Space Sequence World Models, Paul Mattes, Rainer Schlosser, Ralf Herbrich [Paper]

• [2024_041]S4++: Elevating Long Sequence Modeling with State Memory Reply, [Paper]

• [2024_040][Rejected by ICLR 2024] Mamba: Linear-Time Sequence Modeling with Selective State Spaces, Albert Gu, Tri Dao [Paper] [Mamba: The Hard Way] [annotated-mamba]

• [2024_039][ICLR 2024] Gated recurrent neural networks discover attention, Nicolas Zucchet, Seijin Kobayashi, Yassir Akram, Johannes Von Oswald, Maxime Larcher, Angelika Steger, Joao Sacramento [Paper]

• [2024_038][ICLR 2024] GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling, Tobias Katsch [Paper]

• [2024_037][ICLR 2024] Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors, Ido Amos, Jonathan Berant, Ankit Gupta [Paper]

• [2024_036] [ICLR 2024] Mastering Memory Tasks with World Models, Mohammad Reza Samsami, Artem Zholus, Janarthanan Rajendran, Sarath Chandar [Paper] [Project Page] [Code]

• [2024_035]Spectral State Space Models, Naman Agarwal, Daniel Suo, Xinyi Chen, Elad Hazan [Paper]

• [2024_034]Graph Mamba: Towards Learning on Graphs with State Space Models, Ali Behrouz, Farnoosh Hashemi [Paper]

• [2024_033]Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks, Jongho Park, Jaeseung Park, Zheyang Xiong, Nayoung Lee, Jaewoong Cho, Samet Oymak, Kangwook Lee, Dimitris Papailiopoulos [Paper]

• [2024_032]Is Mamba Capable of In-Context Learning? Riccardo Grazzi, Julien Siems, Simon Schrodi, Thomas Brox, Frank Hutter [Paper]

• [2024_031]LOCOST: State-Space Models for Long Document Abstractive Summarization, Florian Le Bronnec, Song Duong, Mathieu Ravaut, Alexandre Allauzen, Nancy F. Chen, Vincent Guigue, Alberto Lumbreras, Laure Soulier, Patrick Gallinari [Paper] [Code]

• [2024_030]RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks, Haowen Hou, F. Richard Yu [Paper] [Code]

• [2024_029]BlackMamba: Mixture of Experts for State-Space Models, Quentin Anthony, Yury Tokpanov, Paolo Glorioso, Beren Millidge [Paper] [Code]

• [2024_028]Recurrent Distance Filtering for Graph Representation Learning, Yuhui Ding, Antonio Orvieto, Bobby He, Thomas Hofmann [Paper] [Code]

• [2024_027]SSM Meets Video Diffusion Models: Efficient Video Generation with Structured State Spaces, Yuta Oshima, Shohei Taniguchi, Masahiro Suzuki, Yutaka Matsuo [Paper]

• [2024_026]Pan-Mamba: Effective pan-sharpening with State Space Model, Xuanhua He, Ke Cao, Keyu Yan, Rui Li, Chengjun Xie, Jie Zhang, Man Zhou [Paper] [Code]

• [2024_025]Weak-Mamba-UNet: Visual Mamba Makes CNN and ViT Work Better for Scribble-based Medical Image Segmentation, Ziyang Wang, Chao Ma [Paper] [Code]

• [2024_024]PointMamba: A Simple State Space Model for Point Cloud Analysis, Dingkang Liang, Xin Zhou, Xinyu Wang, Xingkui Zhu, Wei Xu, Zhikang Zou, Xiaoqing Ye, Xiang Bai [Paper] [Code]

• [2024_023]P-Mamba: Marrying Perona Malik Diffusion with Mamba for Efficient Pediatric Echocardiographic Left Ventricular Segmentation, Zi Ye, Tianxiang Chen [Paper]

• [2024_022]Semi-Mamba-UNet: Pixel-Level Contrastive Cross-Supervised Visual Mamba-based UNet for Semi-Supervised Medical Image Segmentation, Ziyang Wang, Chao Ma [Paper]

• [2024_021]FD-Vision Mamba for Endoscopic Exposure Correction, Zhuoran Zheng, Jun Zhang, [Paper]

• [2024_020]Scalable Diffusion Models with State Space Backbone, Zhengcong Fei, Mingyuan Fan, Changqian Yu, Junshi Huang [Paper] [Code]

• [2024_019]Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data, Shufan Li, Harkanwar Singh, Aditya Grover [Paper]

• [2024_018]Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation, Ziyang Wang, Jian-Qing Zheng, Yichi Zhang, Ge Cui, Lei Li [Paper] [Code]

• [2024_017]MambaTab: A Simple Yet Effective Approach for Handling Tabular Data, Md Atik Ahamed1, Qiang Cheng [Paper]

• [2024_016] nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model, Haifan Gong, Luoyao Kang, Yitao Wang, Xiang Wan, Haofeng Li [Paper] [Code]

• [2024_015] U-shaped Vision Mamba for Single Image Dehazing, Zhuoran Zheng, Chen Wu [Paper]

• [2024_014] Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces, Chloe Wang, Oleksii Tsepa, Jun Ma, Bo Wang [Paper] [Code]

• [2024_013] VM-UNet: Vision Mamba UNet for Medical Image Segmentation, Jiacheng Ruan, Suncheng Xiang [Paper] [Code]

• [2024_012] Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining, Jiarun Liu, Hao Yang, Hong-Yu Zhou, Yan Xi, Lequan Yu, Yizhou Yu, Yong Liang, Guangming Shi, Shaoting Zhang, Hairong Zheng, Shanshan Wang [Paper] [Code]

• [2024_011] Ma, Jun, Feifei Li, and Bo Wang. "U-mamba: Enhancing long-range dependency for biomedical image segmentation." arXiv preprint arXiv:2401.04722 (2024). [Paper] [Code]

• [2024_010] Vivim: a Video Vision Mamba for Medical Video Object Segmentation, Yijun Yang, Zhaohu Xing, and Lei Zhu [Paper] [Code]

• [2024_009] Wang, Junxiong, et al. "MambaByte: Token-free Selective State Space Model." arXiv preprint arXiv:2401.13660 (2024). [Paper] [Code]

• [2024_008] MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts. Pióro, M., Ciebiera, K., Król, K., Ludziejewski, J., & Jaszczur, S. (2024). arXiv preprint arXiv:2401.04081. [Paper]

• [2024_007] [ICLR-2024] MASTERING MEMORY TASKS WITH WORLD MODELS [Paper]

• [2024_006] MambaMorph: a Mamba-based Backbone with Contrastive Feature Learning for Deformable MR-CT Registration, Tao Guo, Yinuo Wang, and Cai Meng [Paper] [Code]

• [2024_005] SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation, [Paper] [Code]

• [2024_003] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model, Lianghui Zhu1∗, Bencheng Liao1∗, Qian Zhang2, Xinlong Wang3, Wenyu Liu1, Xinggang Wang [Paper] [Code]

• [2024_002] VMamba: Visual State Space Model, Yue Liu1,Yunjie Tian1,Yuzhong Zhao1, Hongtian Yu1, Lingxi Xie2, Yaowei Wang3, Qixiang Ye1, Yunfan Liu1 [Paper] [Code]

• [2024_001] Theoretical Foundations of Deep Selective State-Space Models, Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, Terry Lyons [Paper]

Year 2023

• [2023_018] [CHIL 2023] Modeling Multivariate Biosignals With Graph Neural Networks and Structured State Space Models, Siyi Tang, Jared A. Dunnmon, Liangqiong Qu, Khaled K. Saab, Tina Baykaner, Christopher Lee-Messer, Daniel L. Rubin [Paper]

• [2023_017] "StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization." Wang, Shida, and Qianxiao Li. arXiv preprint arXiv:2311.14495 (2023). [Paper] [Code]

• [2023_016] State-space models with layer-wise nonlinearity are universal approximators with exponential decaying memory, Shida Wang, Beichen Xue [Paper]

• [2023_015] Spiking Structured State Space Model for Monaural Speech Enhancement. Du, Y., Liu, X., & Chua, Y. (2023). arXiv preprint arXiv:2309.03641. [Paper]

• [2023_014] Mastering Diverse Domains through World Models, Danijar Hafner,12 Jurgis Pasukonis,1 Jimmy Ba,2 Timothy Lillicrap [Paper] [Code]

• [2023_013] Selective Structured State-Spaces for Long-Form Video Understanding, Jue Wang Wentao Zhu Pichao Wang Xiang Yu Linda Liu Mohamed Omar Raffay Hamid [Paper]

• [2023_012] Mamba: Linear-Time Sequence Modeling with Selective State Spaces, Albert Gu*1and Tri Dao [Paper] [Github]

• [2023_011] [NeurIPS 2023] Structured State Space Models for In-Context Reinforcement Learning, Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, Feryal Behbahani [Paper] [Code]

• [2023_010] Diffusion Models Without Attention, Jing Nathan Yan, Jiatao Gu, Alexander M. Rush [Paper]

• [2023_009] Hierarchically Gated Recurrent Neural Network for Sequence Modeling, Zhen Qin, Songlin Yang, Yiran Zhong [Paper] [Code]

• [2023_008] Retentive Network: A Successor to Transformer for Large Language Models, Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, Furu Wei [Paper] [Code]

• [2023_007] Convolutional State Space Models for Long-Range Spatiotemporal Modeling, Jimmy T.H. Smith, Shalini De Mello, Jan Kautz, Scott W. Linderman, Wonmin Byeon [Paper] [Code]

• [2023_006] Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions, Stefano Massaroli, Michael Poli, Daniel Y. Fu, Hermann Kumbong, Rom N. Parnichkun, Aman Timalsina, David W. Romero, Quinn McIntyre, Beidi Chen, Atri Rudra, Ce Zhang, Christopher Re, Stefano Ermon, Yoshua Bengio [Paper]

• [2023_005] Structured state-space models are deep Wiener models, Fabio Bonassi, Carl Andersson, Per Mattsson, Thomas B. Schön [Paper]

• [2023_004] Zoology: Measuring and Improving Recall in Efficient Language Models, Simran Arora, Sabri Eyuboglu, Aman Timalsina, Isys Johnson, Michael Poli, James Zou, Atri Rudra, Christopher Ré [Paper]

• [2023_003] [ICML 2023] Resurrecting Recurrent Neural Networks for Long Sequences, Antonio Orvieto · Samuel Smith · Albert Gu · Anushan Fernando · Caglar Gulcehre · Razvan Pascanu · Soham De [Paper]

• [2023_002] Hyena Hierarchy: Towards Larger Convolutional Language Models, Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y. Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Stefano Ermon, Christopher Ré [Paper]

• [2023_001] [ICLR 2023] Simplified State Space Layers for Sequence Modeling, Jimmy T.H. Smith, Andrew Warrington, Scott Linderman [Paper]

Year 2022

• [2022_009] [ECCV-2022] Long Movie Clip Classification with State-Space Video Models, Md Mohaiminul Islam, Gedas Bertasius [Paper] [Code]

• [2022_008] [NIPS-2022] "S4nd: Modeling images and videos as multidimensional signals with state spaces." Nguyen, Eric, et al. Advances in neural information processing systems 35 (2022): 2846-2861. [Paper] [Code]

• [2022_007] [Pre-training] Wang, J., Yan, J. N., Gu, A., & Rush, A. M. (2022). Pretraining without attention. arXiv preprint arXiv:2212.10544. [Paper] [Code]

• [2022_006] Long Range Language Modeling via Gated State Spaces, Harsh Mehta1∗ Ankit Gupta2 Ashok Cutkosky3 Behnam Neyshabur1 [Paper]

• [2022_005] [ICML2022] It’s Raw! Audio Generation with State-Space Models, Karan Goel, Albert Gu, Chris Donahue, and Christopher R´e [Paper]

• [2022_004] Diagonal State Spaces are as Effective as Structured State Spaces, Ankit Gupta˚Albert Gu Jonathan Berant [Paper]

• [2022_003] How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections, Albert Gu∗†, Isys Johnson∗‡, Aman Timalsina‡, Atri Rudra‡, and Christopher R´e† [Paper]

• [2022_002] On the Parameterization and Initialization of Diagonal State Space Models, Albert Gu†, Ankit Gupta‡, Karan Goel†, and Christopher R´e† [Paper]

• [2022_001] Efficiently Modeling Long Sequences with Structured State Spaces, Albert Gu, Karan Goel, Christopher Ré [Paper] [The Annotated S4]

Year 2021 and Before

• [2021_003] Efficiently Modeling Long Sequences with Structured State Spaces, Albert Gu, Karan Goel, and Christopher R´e [Paper] [Code]

• [2021_002] HiPPO: Recurrent Memory with Optimal Polynomial Projections, Albert Gu∗†, Tri Dao∗†, Stefano Ermon†, Atri Rudra‡, and Christopher Ré† [Paper] [Code]

• [2021_001] Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers, Albert Gu†, Isys Johnson†, Karan Goel†, Khaled Saab†, Tri Dao†, Atri Rudra‡, and Christopher Ré† [Paper]

Related Models

• Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models, Zhengcong Fei, Mingyuan Fan, Changqian Yu, Debang Li, Junshi Huang [Paper]

• "Retentive network: A successor to transformer for large language models." Sun, Yutao, et al. arXiv preprint arXiv:2307.08621 (2023). [Paper] [Code]

• TLS-RWKV: Real-Time Online Action Detection with Temporal Label Smoothing. Zhu, Z., Shao, W. & Jiao, D. Neural Process Lett 56, 57 (2024). [Paper]

• RRWKV: Capturing Long-range Dependencies in RWKV, Leilei Wang [Paper]

• RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks, Haowen Hou, F. Richard Yu [Paper] [Code]

• Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures, Yuchen Duan, Weiyun Wang, Zhe Chen, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao, Hongsheng Li, Jifeng Dai, Wenhai Wang [Paper] [Code]

• RWKV: Reinventing RNNs for the Transformer Era, Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Jiaju Lin, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Bolun Wang, Johan S. Wind, Stanislaw Wozniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu [Paper]

Other Useful URLs

• [awesome-ssm-ml]
• [Awesome-Mamba-Papers]
• [XiudingCai/Awesome-Mamba-Collection]
• [Awesome-state-space-models] Collection of papers/repos on state-space models.
• [mamba-minimal] Simple, minimal implementation of the Mamba SSM in one file of PyTorch.
• [mamba.py] A simple and efficient Mamba implementation in PyTorch and MLX.
• [Introduction to State Space Models (SSM)]
• [State-Space Modelling by Kevin Kotzé]
• [Structured State Spaces: Combining Continuous-Time, Recurrent, and Convolutional Models]
• [A Visual Guide to Mamba and State Space Models---An Alternative to Transformers for Language Modeling] FEB 19, 2024, by MAARTEN GROOTENDORST.
• [Structured State Spaces: A Brief Survey of Related Models] by Albert Gu, Karan Goel, Khaled Saab, and Chris Ré
• [Video-Tutorial] [Mamba and S4 Explained: Architecture, Parallel Scan, Kernel Fusion, Recurrent, Convolution, Math] by Umar Jamil [Mamba_Slides.pdf]

:newspaper: Citation

If you think this survey is helpful, please feel free to leave a star ⭐️ and cite our paper:

@misc{Wang2024SSMSurvey,
      title={State Space Model for New-Generation Network Alternative to Transformers: A Survey}, 
      author={Xiao Wang and Shiao Wang and Yuhe Ding and Yuehang Li and Wentao Wu and Yao Rong and Weizhe Kong and Ju Huang and Shihao Li and Haoxiang Yang and Ziwen Wang and Bo Jiang and Chenglong Li and Yaowei Wang and Yonghong Tian and Jin Tang},
      year={2024},
      eprint={2404.09516},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Star History