⚡ Build your chatbot within minutes on your favorite device; offer SOTA...
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
scalable and robust tree-based speculative decoding algorithm
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarc...
[NeurIPS'23] Speculative Decoding with Big Little Decoder