Alpaca Eval Versions Save

What's Changed

• [WIP] GPT4 turbo as evaluator by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/160
• [ENH] add GPT4 turbo as evaluator in README by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/165
• Add minichat-3b to AlpacaEval by @GeneZC in https://github.com/tatsu-lab/alpaca_eval/pull/167
• fix: filter openai spam filter by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/169

New Contributors

• @GeneZC made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/167

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.3.3...vv0.3.4

What's Changed

• Gpt4 turbo by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/159

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.3.2...v0.3.3

What's Changed

• add UltraLM-13b-V2.0/UltraLM-13b-V2.0-best-of-16/UltraLM-13b-best-of-16 to AlpacaEval by @lifan-yuan in https://github.com/tatsu-lab/alpaca_eval/pull/139
• Add annotations & fix leaderboard by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/142
• refresh Cohere by @sanderland in https://github.com/tatsu-lab/alpaca_eval/pull/141
• Add PlatoLM-7B to AlpacaEval by @renatz in https://github.com/tatsu-lab/alpaca_eval/pull/143
• Add evo-7b to AlpacaEval by @zfang in https://github.com/tatsu-lab/alpaca_eval/pull/144
• Add NEFTune models to AlpacaEval by @neelsjain in https://github.com/tatsu-lab/alpaca_eval/pull/146
• Add claude2-alpaca-13b, recycled-wizardlm-7b-v1.0, recycled-wizardlm-… by @MingLiiii in https://github.com/tatsu-lab/alpaca_eval/pull/147
• Add CausalLM/14B to AlpacaEval by @CausalLM in https://github.com/tatsu-lab/alpaca_eval/pull/148
• Add Zephyr 7B evals by @lewtun in https://github.com/tatsu-lab/alpaca_eval/pull/152
• Add Evo v2 7B by @zfang in https://github.com/tatsu-lab/alpaca_eval/pull/153
• Add decoder for calling Anthropic models via Amazon Bedrock by @billcai in https://github.com/tatsu-lab/alpaca_eval/pull/151
• cohere update by @sanderland in https://github.com/tatsu-lab/alpaca_eval/pull/155
• feat: upgrade to openai 1.0.0 by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/157

New Contributors

• @lifan-yuan made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/139
• @renatz made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/143
• @zfang made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/144
• @neelsjain made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/146
• @MingLiiii made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/147
• @CausalLM made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/148
• @lewtun made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/152
• @billcai made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/151

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.3.1...v0.3.2

What's Changed

• Add results of Xwin-LM by @nbl97 in https://github.com/tatsu-lab/alpaca_eval/pull/135
• [ENH] add gpt 3.5 instruct by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/137

New Contributors

• @nbl97 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/135

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.3.0...v0.3.1

What's Changed

• [ENH] add fixed gpt4 version annotator by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/127
• Add openbuddy-llama2-13b-v11.1 by @44670 in https://github.com/tatsu-lab/alpaca_eval/pull/129
• [ENH] add max concurrency oai by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/131

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.2.9...v0.3.0

What's Changed

• Ensure primary keys are string & decrease processes for OpenAI by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/116
• Add JinaChat to the leaderboards by @jupyterjazz in https://github.com/tatsu-lab/alpaca_eval/pull/117
• [BUG] jina chat error in configs by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/118
• Add Humpback to AlpacaEval by @xianxl in https://github.com/tatsu-lab/alpaca_eval/pull/120
• update Humpback results by @xianxl in https://github.com/tatsu-lab/alpaca_eval/pull/121
• add link to Humpback paper by @xianxl in https://github.com/tatsu-lab/alpaca_eval/pull/122
• Add vllm decoder for model inference by @44670 in https://github.com/tatsu-lab/alpaca_eval/pull/124
• [ENH] return completions_all and allow sequence of max_tokens by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/125

New Contributors

• @jupyterjazz made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/117
• @xianxl made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/120

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.2.8...v0.2.9

What's Changed

• [BUG] closes #77 by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/109
• Add openbuddy-llama-30b-v7.1 to AlpacaEval by @44670 in https://github.com/tatsu-lab/alpaca_eval/pull/108
• Fix typo on pretty_name by @44670 in https://github.com/tatsu-lab/alpaca_eval/pull/110
• Add openbuddy-falcon-40b-v9 to AlpacaEval by @44670 in https://github.com/tatsu-lab/alpaca_eval/pull/111
• [CLEAN] remove warning by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/112
• [BUG] utils.DUMMY_EXAMPLE by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/113

New Contributors

• @44670 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/108

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.2.7...v0.2.8

What's Changed

• Update WizardLM 13B V1.2 results by @victorsungo in https://github.com/tatsu-lab/alpaca_eval/pull/99
• [ENH] llama70B and chunking by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/100
• [ENH] add pipeline meta parser by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/103
• [CLEAN] Single annotator not abstract by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/104
• Add OpenChat 3.1 Results by @imoneoi in https://github.com/tatsu-lab/alpaca_eval/pull/105
• [ENH] add example with HF API by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/106

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.2.6...v0.2.7

What's Changed

• [STYLE] fix ill-formatted logging message by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/97
• [STYLE] PR medium eval (ANNOTATOR_COLUMN) by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/98

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.2.5...v0.2.6

What's Changed

• [ENH] adds processors by @YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/95

Full Changelog: https://github.com/tatsu-lab/alpaca_eval/compare/v0.2.4...v0.2.5