Official release of InternLM2 7B and 20B base and chat models. 200K cont...
Code and documents of LongLoRA and LongAlpaca
Transformers with Arbitrarily Large Context
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in P...
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in ...
The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs ...
Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 10...
PyTorch implementation of Infini-Transformer from "Leave No Context Behi...
LongQLoRA: Extent Context Length of LLMs Efficiently
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarc...
LooGLE: Long Context Evaluation for Long-Context Language Models
LongAlign: A Recipe for Long Context Alignment Encompassing Data, Traini...
The official repo for "LLoCo: Learning Long Contexts Offline"
Implementation of Perceiver AR, Deepmind's new long-context attention ne...