Ragas Versions Save

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines

v0.1.7

1 month ago

What's Changed

minor: fix llm expansion in docs by @glenacota in https://github.com/explodinggradients/ragas/pull/847
Update llama_index imports by @sestinj in https://github.com/explodinggradients/ragas/pull/849
feat: Change parsers and add custom critic model by @shahules786 in https://github.com/explodinggradients/ragas/pull/838
fix: change documentation to reflect accurate method name by @dbtreasure in https://github.com/explodinggradients/ragas/pull/837
Update answer_relevance.md by @carllapierre in https://github.com/explodinggradients/ragas/pull/832
fix ci: some linting issues by @jjmachan in https://github.com/explodinggradients/ragas/pull/852

New Contributors

@glenacota made their first contribution in https://github.com/explodinggradients/ragas/pull/847
@sestinj made their first contribution in https://github.com/explodinggradients/ragas/pull/849
@dbtreasure made their first contribution in https://github.com/explodinggradients/ragas/pull/837
@carllapierre made their first contribution in https://github.com/explodinggradients/ragas/pull/832

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.6...v0.1.7

v0.1.6

1 month ago

What's Changed

Fix: json format inconsistency in context recall (prompt for statemen… by @jokokojote in https://github.com/explodinggradients/ragas/pull/776
fix: Explanaitions typo on index.md by @dcamejo1 in https://github.com/explodinggradients/ragas/pull/801
feat(integrations): add support for langchain and langsmith by @jjmachan in https://github.com/explodinggradients/ragas/pull/795
Docs: fix confusion regarding getting started by @shahules786 in https://github.com/explodinggradients/ragas/pull/815
docs: add openlayer integration by @shahules786 in https://github.com/explodinggradients/ragas/pull/827
fix: Eventloop unclosed by @yunfwe in https://github.com/explodinggradients/ragas/pull/796
More robust JSON prompting and parsing by @mrtj in https://github.com/explodinggradients/ragas/pull/807
fix(integration): langchain integration and CI fixes by @jjmachan in https://github.com/explodinggradients/ragas/pull/830
Added more examples to the translation prompts by @baswenneker in https://github.com/explodinggradients/ragas/pull/826
fix: use aparse in all metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/831

New Contributors

@dcamejo1 made their first contribution in https://github.com/explodinggradients/ragas/pull/801
@yunfwe made their first contribution in https://github.com/explodinggradients/ragas/pull/796
@mrtj made their first contribution in https://github.com/explodinggradients/ragas/pull/807
@baswenneker made their first contribution in https://github.com/explodinggradients/ragas/pull/826

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.5...v0.1.6

v0.1.5

1 month ago

What's Changed

fix: handle string score in node filter by @shahules786 in https://github.com/explodinggradients/ragas/pull/755
Add ipython notebook demonstrating the ragas-haystack integration. by @annthurium in https://github.com/explodinggradients/ragas/pull/754
Update Bring Your Own LLM Docs Link by @pmbaumgartner in https://github.com/explodinggradients/ragas/pull/765
fix: edge case in answer_correctness by @shahules786 in https://github.com/explodinggradients/ragas/pull/778
Added metadata field to datarows by @rnbokade in https://github.com/explodinggradients/ragas/pull/782
fix: is_async not passed for llm generation in context recall by @jokokojote in https://github.com/explodinggradients/ragas/pull/777
fix: Nan in metrics/test set gen by @shahules786 in https://github.com/explodinggradients/ragas/pull/786
Fix adapt issue for keyphrase extractor by @smodlich in https://github.com/explodinggradients/ragas/pull/766
updated context_scoring_prompt and provide extra examples. by @nofrish in https://github.com/explodinggradients/ragas/pull/725
fix: faithfulness by @shahules786 in https://github.com/explodinggradients/ragas/pull/787

New Contributors

@annthurium made their first contribution in https://github.com/explodinggradients/ragas/pull/754
@rnbokade made their first contribution in https://github.com/explodinggradients/ragas/pull/782
@smodlich made their first contribution in https://github.com/explodinggradients/ragas/pull/766
@nofrish made their first contribution in https://github.com/explodinggradients/ragas/pull/725

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.4...v0.1.5

v0.1.4

2 months ago

What's Changed

docs: Add docs for new entity based metric by @sky-2002 in https://github.com/explodinggradients/ragas/pull/674
Fix typos that cause fatal assertion errors in prompts by @joy13975 in https://github.com/explodinggradients/ragas/pull/676
fix: edge case in faithfulness by @shahules786 in https://github.com/explodinggradients/ragas/pull/682
docs: Fix the precision calculation in context-precision metric doc by @amit-timalsina in https://github.com/explodinggradients/ragas/pull/685
fix: improving reproducability of evals when running everything together by @jjmachan in https://github.com/explodinggradients/ragas/pull/671
added octlane docs by @jjmachan in https://github.com/explodinggradients/ragas/pull/695
Docs: Fix content precision description by @jokokojote in https://github.com/explodinggradients/ragas/pull/693
Docs: Created own section for Calculation (instead of Dropdown) by @GeroVanMi in https://github.com/explodinggradients/ragas/pull/696
Fix: Enable Critique Prompt Customization in AspectCritique by @AaronKim218 in https://github.com/explodinggradients/ragas/pull/700
chore: improve bug/question template by @shahules786 in https://github.com/explodinggradients/ragas/pull/702
docs: clean and smooth by @shahules786 in https://github.com/explodinggradients/ragas/pull/703
Docs: Fix and supplement answer relevancy description by @jokokojote in https://github.com/explodinggradients/ragas/pull/705
fix: remove unwanted quotes from questions by @shahules786 in https://github.com/explodinggradients/ragas/pull/708
Readme Formatting by @gkamradt in https://github.com/explodinggradients/ragas/pull/710
docs: customising embeddings and llms by @shahules786 in https://github.com/explodinggradients/ragas/pull/712
chore: add copyright by @shahules786 in https://github.com/explodinggradients/ragas/pull/716
feat: test set generation improvements by @shahules786 in https://github.com/explodinggradients/ragas/pull/719
Adding Tonic Validate integration notebook by @akamor in https://github.com/explodinggradients/ragas/pull/707
fix: edge case in CR by @shahules786 in https://github.com/explodinggradients/ragas/pull/728
fix(ragas): remove mutable defaults by @mspronesti in https://github.com/explodinggradients/ragas/pull/684
fix: remove experimental filters by @shahules786 in https://github.com/explodinggradients/ragas/pull/743
Correct Faithfulness Verdict Map by @pmbaumgartner in https://github.com/explodinggradients/ragas/pull/739
feat(generator): extend construction to any langchain LLM and Embeddings by @mspronesti in https://github.com/explodinggradients/ragas/pull/670
docs: minor corrections by @shahules786 in https://github.com/explodinggradients/ragas/pull/747

New Contributors

@amit-timalsina made their first contribution in https://github.com/explodinggradients/ragas/pull/685
@jokokojote made their first contribution in https://github.com/explodinggradients/ragas/pull/693
@GeroVanMi made their first contribution in https://github.com/explodinggradients/ragas/pull/696
@AaronKim218 made their first contribution in https://github.com/explodinggradients/ragas/pull/700
@gkamradt made their first contribution in https://github.com/explodinggradients/ragas/pull/710
@akamor made their first contribution in https://github.com/explodinggradients/ragas/pull/707
@mspronesti made their first contribution in https://github.com/explodinggradients/ragas/pull/684
@pmbaumgartner made their first contribution in https://github.com/explodinggradients/ragas/pull/739

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.3...v0.1.4

v0.1.3

2 months ago

What's Changed

feat: improved node selection and question fixing by @shahules786 in https://github.com/explodinggradients/ragas/pull/644
docs: add explanation to metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/658
fix: edge cases in metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/661
Update evolution by @cuent in https://github.com/explodinggradients/ragas/pull/666
feat: Add new "entity-based context recall" metric by @sky-2002 in https://github.com/explodinggradients/ragas/pull/667
fix: filter and context selection by @shahules786 in https://github.com/explodinggradients/ragas/pull/672
Add support for optional max concurrency by @joy13975 in https://github.com/explodinggradients/ragas/pull/643

New Contributors

@cuent made their first contribution in https://github.com/explodinggradients/ragas/pull/666
@sky-2002 made their first contribution in https://github.com/explodinggradients/ragas/pull/667

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.2...v0.1.3

v0.1.2

2 months ago

What's Changed

docs: arize/ ragas notebook by @axiomofjoy in https://github.com/explodinggradients/ragas/pull/594
fix: prompt adaptation type checks by @shahules786 in https://github.com/explodinggradients/ragas/pull/615
fix: node indexing in testset generator by @shahules786 in https://github.com/explodinggradients/ragas/pull/617
Update ragas-arize.ipynb by @PubliusAu in https://github.com/explodinggradients/ragas/pull/618
docs: getting started by @jjmachan in https://github.com/explodinggradients/ragas/pull/589
Added testset generation for bedrock by @VpkPrasanna in https://github.com/explodinggradients/ragas/pull/626
Made embeddings and LLMs dependent on metric in evaluate function by @peanutshawny in https://github.com/explodinggradients/ragas/pull/628
docs: bring your own llms and embeddings by @jjmachan in https://github.com/explodinggradients/ragas/pull/631
docs: fix import in the example code: ContextRelevance -> ContextRelevancy by @izikeros in https://github.com/explodinggradients/ragas/pull/646
Fix typo: Teset -> Testset by @joy13975 in https://github.com/explodinggradients/ragas/pull/638
Update ragas-arize.ipynb by @PubliusAu in https://github.com/explodinggradients/ragas/pull/637
renaming ground_truths to ground_truth. by @alexjyong in https://github.com/explodinggradients/ragas/pull/640
fix: added types so you can pass langchain llms and embeddings too by @jjmachan in https://github.com/explodinggradients/ragas/pull/652

New Contributors

@axiomofjoy made their first contribution in https://github.com/explodinggradients/ragas/pull/594
@PubliusAu made their first contribution in https://github.com/explodinggradients/ragas/pull/618
@VpkPrasanna made their first contribution in https://github.com/explodinggradients/ragas/pull/626
@peanutshawny made their first contribution in https://github.com/explodinggradients/ragas/pull/628
@izikeros made their first contribution in https://github.com/explodinggradients/ragas/pull/646
@joy13975 made their first contribution in https://github.com/explodinggradients/ragas/pull/638
@alexjyong made their first contribution in https://github.com/explodinggradients/ragas/pull/640

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.1...v0.1.2

v0.1.1

3 months ago

What's Changed

Fixing typos, vertextai to vertex ai by @mrchtr in https://github.com/explodinggradients/ragas/pull/572
fix typo by @Kyeongpil in https://github.com/explodinggradients/ragas/pull/570
fix assertions and quality in prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/574
fix: ensure dict type by @shahules786 in https://github.com/explodinggradients/ragas/pull/578
style: remove additional comments in prompt.py by @leehanchung in https://github.com/explodinggradients/ragas/pull/577
Add stepkurniawan community contribution by @stepkurniawan in https://github.com/explodinggradients/ragas/pull/585
Parse JSON string in adapt() by @mitmul in https://github.com/explodinggradients/ragas/pull/584
fix: replace string formating by dict by @shahules786 in https://github.com/explodinggradients/ragas/pull/588
[patch] Reduce NaN Occurrences by Simple Prompt Modification for JSON Output for context_precision by @i-w-a in https://github.com/explodinggradients/ragas/pull/581
ci: migrate to latest llama-index version by @shahules786 in https://github.com/explodinggradients/ragas/pull/603
Fix: evolution flows for test generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/602
fix: llamaIndex latest version by @jjmachan in https://github.com/explodinggradients/ragas/pull/604
Chore: add language to event tracking by @shahules786 in https://github.com/explodinggradients/ragas/pull/592
feat(llms.json_load): Recursively load json lists by @pberger514 in https://github.com/explodinggradients/ragas/pull/593

New Contributors

@mrchtr made their first contribution in https://github.com/explodinggradients/ragas/pull/572
@Kyeongpil made their first contribution in https://github.com/explodinggradients/ragas/pull/570
@leehanchung made their first contribution in https://github.com/explodinggradients/ragas/pull/577
@stepkurniawan made their first contribution in https://github.com/explodinggradients/ragas/pull/585
@mitmul made their first contribution in https://github.com/explodinggradients/ragas/pull/584
@i-w-a made their first contribution in https://github.com/explodinggradients/ragas/pull/581
@pberger514 made their first contribution in https://github.com/explodinggradients/ragas/pull/593

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.0...v0.1.1

v0.1.0

3 months ago

What's Changed

add initial embeddings.encode_kwargs by @lucasiscovici in https://github.com/explodinggradients/ragas/pull/371
feat(testset_generator): support error catching in generation process by @yuukidach in https://github.com/explodinggradients/ragas/pull/368
added class for FastEmbed by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/379
Updated NLI Verdicts to Binary Format by @lucasiscovici in https://github.com/explodinggradients/ragas/pull/384
added docs for ragas embeddings by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/386
fix context in _answer_correctness.py by @lucasiscovici in https://github.com/explodinggradients/ragas/pull/385
docs: AI copilot implementation by @nickscamara in https://github.com/explodinggradients/ragas/pull/294
fix: faithfulness for evasive/reject answers by @shahules786 in https://github.com/explodinggradients/ragas/pull/392
Added prompt class by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/388
Context precision with ground truth by @shahules786 in https://github.com/explodinggradients/ragas/pull/391
Added note to answer_relevance by @shahules786 in https://github.com/explodinggradients/ragas/pull/399
Import evaluate function before patching RagasLLM, fixes #301 by @HySoaKa in https://github.com/explodinggradients/ragas/pull/377
Fix: Added ground_truths arg type-hint info in the eval function by @MANISH007700 in https://github.com/explodinggradients/ragas/pull/404
Typo and phrasing correction by @TruscaPetre in https://github.com/explodinggradients/ragas/pull/408
Small improvements to documentation custom LLMs by @TruscaPetre in https://github.com/explodinggradients/ragas/pull/410
feat(metrics): skip embedding process when the corresponding weight is zero by @yuukidach in https://github.com/explodinggradients/ragas/pull/409
added prompt objects to metrics by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/401
feat: Automatic Prompt adaptation [language] by @shahules786 in https://github.com/explodinggradients/ragas/pull/407
Update _faithfulness.py by @Robs1999 in https://github.com/explodinggradients/ragas/pull/416
fix(metrics): avoid dividing by zero in correctness by @yuukidach in https://github.com/explodinggradients/ragas/pull/421
Feature/doc update ragas score by @prufrock349 in https://github.com/explodinggradients/ragas/pull/417
Docs: Prompts and prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/419
fix(metrics): wrong usage of prompt by @yuukidach in https://github.com/explodinggradients/ragas/pull/422
fix: context_utilization name override by @jjmachan in https://github.com/explodinggradients/ragas/pull/424
feat/executor: make ragas faster and more robust to run by @jjmachan in https://github.com/explodinggradients/ragas/pull/390
fix: Import PrompValue by @shahules786 in https://github.com/explodinggradients/ragas/pull/427
fix: add uuid to analytics by @shahules786 in https://github.com/explodinggradients/ragas/pull/428
feat: added async version for json_load by @jjmachan in https://github.com/explodinggradients/ragas/pull/430
fix: comment error in analytics by @jjmachan in https://github.com/explodinggradients/ragas/pull/433
fix: payload in analytics by @shahules786 in https://github.com/explodinggradients/ragas/pull/434
fix: tests for analytics by @jjmachan in https://github.com/explodinggradients/ragas/pull/435
Docs: tracing and logging by @shahules786 in https://github.com/explodinggradients/ragas/pull/436
feat: global prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/432
chore: remove mac artifacts from store by @jjmachan in https://github.com/explodinggradients/ragas/pull/438
fix: vary temperature based on strictness by @shahules786 in https://github.com/explodinggradients/ragas/pull/437
docs: custom llms by @shahules786 in https://github.com/explodinggradients/ragas/pull/442
feat: improving testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/381
Fix: context_recall AttributeError by @0ENZO in https://github.com/explodinggradients/ragas/pull/453
Update faithfulness.md by @francesco-defalco in https://github.com/explodinggradients/ragas/pull/451
Fixing typo in faithfulness by @jamesbraza in https://github.com/explodinggradients/ragas/pull/450
fix: context_recall example by @shahules786 in https://github.com/explodinggradients/ragas/pull/458
fix: answer relevancy parsing by @shahules786 in https://github.com/explodinggradients/ragas/pull/461
feat: MVP for the new TestsetGenerator - SimpleEvolution by @jjmachan in https://github.com/explodinggradients/ragas/pull/464
fix: clean up embeddings for ragas and add docs for azure embeddings by @jjmachan in https://github.com/explodinggradients/ragas/pull/477
fix: polishing raise_exception and max_workers by @jjmachan in https://github.com/explodinggradients/ragas/pull/486
Convert prompts in testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/484
feat(testset generation): implement new TestsetGenerator by @jjmachan in https://github.com/explodinggradients/ragas/pull/487
doc: add community page by @shahules786 in https://github.com/explodinggradients/ragas/pull/488
Fix ValueError: diag requires an array of at least two dimensions #474 by @herman159 in https://github.com/explodinggradients/ragas/pull/475
doc: modify prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/490
docs: theming community by @jjmachan in https://github.com/explodinggradients/ragas/pull/491
fix: CP example by @shahules786 in https://github.com/explodinggradients/ragas/pull/489
feat(testset): ported simple, multi_context and reasoning evolutions by @jjmachan in https://github.com/explodinggradients/ragas/pull/496
fix: cleanup old testset generator by @jjmachan in https://github.com/explodinggradients/ragas/pull/500
feat: added tracking of minimal stuff in testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/509
feat: added generate_from_langchain function by @jjmachan in https://github.com/explodinggradients/ragas/pull/511
docs: updated testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/510
fix: answer_correctness embedding by @jjmachan in https://github.com/explodinggradients/ragas/pull/513
added docs for customisation by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/514
fix: added rate_limter to langchain wrapper by @jjmachan in https://github.com/explodinggradients/ragas/pull/517
Update testset_generation.md by @Uzarel in https://github.com/explodinggradients/ragas/pull/516
fix: reshape matrix by @shahules786 in https://github.com/explodinggradients/ragas/pull/519
fix (metrics): changed ground_truths -> ground_truth by @jjmachan in https://github.com/explodinggradients/ragas/pull/521
docs: fix spelling by @shahules786 in https://github.com/explodinggradients/ragas/pull/522
fix: removed old prompt by @shahules786 in https://github.com/explodinggradients/ragas/pull/523
feat(testset): added conditional evolution by @jjmachan in https://github.com/explodinggradients/ragas/pull/524
fix: derive answer from relevant context by @shahules786 in https://github.com/explodinggradients/ragas/pull/525
feat(executor): remove the need to have both sync and async versions of the function by @jjmachan in https://github.com/explodinggradients/ragas/pull/527
feat: seed questions using keyphrases by @shahules786 in https://github.com/explodinggradients/ragas/pull/526
docs: updated contributors by @jjmachan in https://github.com/explodinggradients/ragas/pull/532
feat: configure retries and timeouts with evaluations and testset generators by @jjmachan in https://github.com/explodinggradients/ragas/pull/534
Fix case for few shots of context precision metric by @HuskyDanny in https://github.com/explodinggradients/ragas/pull/541
Updated imports, added dotenv by @aevedis in https://github.com/explodinggradients/ragas/pull/539
feat(executor): run executor in different thread to keep things clean by @jjmachan in https://github.com/explodinggradients/ragas/pull/544
add normalization factor to answer similarity score calculation by @enestydarealmc in https://github.com/explodinggradients/ragas/pull/538
Feat: prompt adaptation for test data generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/530
fix:handle non dict output by @shahules786 in https://github.com/explodinggradients/ragas/pull/547
docs: adaptation for test set generator by @shahules786 in https://github.com/explodinggradients/ragas/pull/548
Fix: ensure quality of adapted prompts by @shahules786 in https://github.com/explodinggradients/ragas/pull/546
fix: raise warning for invalid JSON by @shahules786 in https://github.com/explodinggradients/ragas/pull/550
docs: adding code highlighting by @jjmachan in https://github.com/explodinggradients/ragas/pull/552
fix: wrap langchain embeddings automatically by @jjmachan in https://github.com/explodinggradients/ragas/pull/553
Added documentation page: Using Ragas with Athina for production performance metrics by @shivsak in https://github.com/explodinggradients/ragas/pull/558
fix: proper error message and fixed hf embedding error by @jjmachan in https://github.com/explodinggradients/ragas/pull/561
fix: answer_correctness doesn't reset stuff properly by @jjmachan in https://github.com/explodinggradients/ragas/pull/562
docs: fix issue template by @jjmachan in https://github.com/explodinggradients/ragas/pull/565
docs: fix issue template by @jjmachan in https://github.com/explodinggradients/ragas/pull/566
fix: handle non dict cases by @shahules786 in https://github.com/explodinggradients/ragas/pull/564
fix wrong type annotations and nonexistent types by @stydxm in https://github.com/explodinggradients/ragas/pull/560
fix: raise ExceptionInRunner if executor faces any issues by @jjmachan in https://github.com/explodinggradients/ragas/pull/569

New Contributors

@lucasiscovici made their first contribution in https://github.com/explodinggradients/ragas/pull/371
@nickscamara made their first contribution in https://github.com/explodinggradients/ragas/pull/294
@HySoaKa made their first contribution in https://github.com/explodinggradients/ragas/pull/377
@TruscaPetre made their first contribution in https://github.com/explodinggradients/ragas/pull/408
@Robs1999 made their first contribution in https://github.com/explodinggradients/ragas/pull/416
@prufrock349 made their first contribution in https://github.com/explodinggradients/ragas/pull/417
@0ENZO made their first contribution in https://github.com/explodinggradients/ragas/pull/453
@francesco-defalco made their first contribution in https://github.com/explodinggradients/ragas/pull/451
@jamesbraza made their first contribution in https://github.com/explodinggradients/ragas/pull/450
@herman159 made their first contribution in https://github.com/explodinggradients/ragas/pull/475
@Uzarel made their first contribution in https://github.com/explodinggradients/ragas/pull/516
@HuskyDanny made their first contribution in https://github.com/explodinggradients/ragas/pull/541
@aevedis made their first contribution in https://github.com/explodinggradients/ragas/pull/539
@enestydarealmc made their first contribution in https://github.com/explodinggradients/ragas/pull/538
@shivsak made their first contribution in https://github.com/explodinggradients/ragas/pull/558
@stydxm made their first contribution in https://github.com/explodinggradients/ragas/pull/560

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.0.22...v0.1.0

v0.1.0rc1

3 months ago

What's Changed

add initial embeddings.encode_kwargs by @lucasiscovici in https://github.com/explodinggradients/ragas/pull/371
feat(testset_generator): support error catching in generation process by @yuukidach in https://github.com/explodinggradients/ragas/pull/368
added class for FastEmbed by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/379
Updated NLI Verdicts to Binary Format by @lucasiscovici in https://github.com/explodinggradients/ragas/pull/384
added docs for ragas embeddings by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/386
fix context in _answer_correctness.py by @lucasiscovici in https://github.com/explodinggradients/ragas/pull/385
docs: AI copilot implementation by @nickscamara in https://github.com/explodinggradients/ragas/pull/294
fix: faithfulness for evasive/reject answers by @shahules786 in https://github.com/explodinggradients/ragas/pull/392
Added prompt class by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/388
Context precision with ground truth by @shahules786 in https://github.com/explodinggradients/ragas/pull/391
Added note to answer_relevance by @shahules786 in https://github.com/explodinggradients/ragas/pull/399
Import evaluate function before patching RagasLLM, fixes #301 by @HySoaKa in https://github.com/explodinggradients/ragas/pull/377
Fix: Added ground_truths arg type-hint info in the eval function by @MANISH007700 in https://github.com/explodinggradients/ragas/pull/404
Typo and phrasing correction by @TruscaPetre in https://github.com/explodinggradients/ragas/pull/408
Small improvements to documentation custom LLMs by @TruscaPetre in https://github.com/explodinggradients/ragas/pull/410
feat(metrics): skip embedding process when the corresponding weight is zero by @yuukidach in https://github.com/explodinggradients/ragas/pull/409
added prompt objects to metrics by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/401
feat: Automatic Prompt adaptation [language] by @shahules786 in https://github.com/explodinggradients/ragas/pull/407
Update _faithfulness.py by @Robs1999 in https://github.com/explodinggradients/ragas/pull/416
fix(metrics): avoid dividing by zero in correctness by @yuukidach in https://github.com/explodinggradients/ragas/pull/421
Feature/doc update ragas score by @prufrock349 in https://github.com/explodinggradients/ragas/pull/417
Docs: Prompts and prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/419
fix(metrics): wrong usage of prompt by @yuukidach in https://github.com/explodinggradients/ragas/pull/422
fix: context_utilization name override by @jjmachan in https://github.com/explodinggradients/ragas/pull/424
feat/executor: make ragas faster and more robust to run by @jjmachan in https://github.com/explodinggradients/ragas/pull/390
fix: Import PrompValue by @shahules786 in https://github.com/explodinggradients/ragas/pull/427
fix: add uuid to analytics by @shahules786 in https://github.com/explodinggradients/ragas/pull/428
feat: added async version for json_load by @jjmachan in https://github.com/explodinggradients/ragas/pull/430
fix: comment error in analytics by @jjmachan in https://github.com/explodinggradients/ragas/pull/433
fix: payload in analytics by @shahules786 in https://github.com/explodinggradients/ragas/pull/434
fix: tests for analytics by @jjmachan in https://github.com/explodinggradients/ragas/pull/435
Docs: tracing and logging by @shahules786 in https://github.com/explodinggradients/ragas/pull/436
feat: global prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/432
chore: remove mac artifacts from store by @jjmachan in https://github.com/explodinggradients/ragas/pull/438
fix: vary temperature based on strictness by @shahules786 in https://github.com/explodinggradients/ragas/pull/437
docs: custom llms by @shahules786 in https://github.com/explodinggradients/ragas/pull/442
feat: improving testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/381
Fix: context_recall AttributeError by @0ENZO in https://github.com/explodinggradients/ragas/pull/453
Update faithfulness.md by @francesco-defalco in https://github.com/explodinggradients/ragas/pull/451
Fixing typo in faithfulness by @jamesbraza in https://github.com/explodinggradients/ragas/pull/450
fix: context_recall example by @shahules786 in https://github.com/explodinggradients/ragas/pull/458
fix: answer relevancy parsing by @shahules786 in https://github.com/explodinggradients/ragas/pull/461
feat: MVP for the new TestsetGenerator - SimpleEvolution by @jjmachan in https://github.com/explodinggradients/ragas/pull/464
fix: clean up embeddings for ragas and add docs for azure embeddings by @jjmachan in https://github.com/explodinggradients/ragas/pull/477
fix: polishing raise_exception and max_workers by @jjmachan in https://github.com/explodinggradients/ragas/pull/486
Convert prompts in testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/484
feat(testset generation): implement new TestsetGenerator by @jjmachan in https://github.com/explodinggradients/ragas/pull/487
doc: add community page by @shahules786 in https://github.com/explodinggradients/ragas/pull/488
Fix ValueError: diag requires an array of at least two dimensions #474 by @herman159 in https://github.com/explodinggradients/ragas/pull/475
doc: modify prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/490
docs: theming community by @jjmachan in https://github.com/explodinggradients/ragas/pull/491
fix: CP example by @shahules786 in https://github.com/explodinggradients/ragas/pull/489
feat(testset): ported simple, multi_context and reasoning evolutions by @jjmachan in https://github.com/explodinggradients/ragas/pull/496
fix: cleanup old testset generator by @jjmachan in https://github.com/explodinggradients/ragas/pull/500
feat: added tracking of minimal stuff in testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/509
feat: added generate_from_langchain function by @jjmachan in https://github.com/explodinggradients/ragas/pull/511
docs: updated testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/510
fix: answer_correctness embedding by @jjmachan in https://github.com/explodinggradients/ragas/pull/513

New Contributors

@lucasiscovici made their first contribution in https://github.com/explodinggradients/ragas/pull/371
@nickscamara made their first contribution in https://github.com/explodinggradients/ragas/pull/294
@HySoaKa made their first contribution in https://github.com/explodinggradients/ragas/pull/377
@TruscaPetre made their first contribution in https://github.com/explodinggradients/ragas/pull/408
@Robs1999 made their first contribution in https://github.com/explodinggradients/ragas/pull/416
@prufrock349 made their first contribution in https://github.com/explodinggradients/ragas/pull/417
@0ENZO made their first contribution in https://github.com/explodinggradients/ragas/pull/453
@francesco-defalco made their first contribution in https://github.com/explodinggradients/ragas/pull/451
@jamesbraza made their first contribution in https://github.com/explodinggradients/ragas/pull/450
@herman159 made their first contribution in https://github.com/explodinggradients/ragas/pull/475

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.0.22...v0.1.0rc1

v0.0.22

5 months ago

What's Changed

docs: comparing llms using ragas evaluations by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/314
convert context precision to json format by @shahules786 in https://github.com/explodinggradients/ragas/pull/322
fix: faithfulness.md by @kaiyikang in https://github.com/explodinggradients/ragas/pull/328
fix: structure faithfulness output by @shahules786 in https://github.com/explodinggradients/ragas/pull/333
fix: Column remap fix for Dataset evaluation by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/331
Fix OpenAIEmbeddings reference error in answer_relevancy by @ajinkyavbhandare in https://github.com/explodinggradients/ragas/pull/329
Fix: minor improvements across metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/337
fix wrong number by @Yongtae723 in https://github.com/explodinggradients/ragas/pull/338
fix: faithfulness for unrelated answering by @shahules786 in https://github.com/explodinggradients/ragas/pull/340
fix: load response as json by @shahules786 in https://github.com/explodinggradients/ragas/pull/341
docs: remove answer_relevancy from llm-docs by @tinomaxthayil in https://github.com/explodinggradients/ragas/pull/343
fix: bedrock llm error when evaluating rag qa validate_api_key by @arm-diaz in https://github.com/explodinggradients/ragas/pull/350
Added integration with Amazon API Gateway + bug fix in _answer_relevance.py by @brunopistone in https://github.com/explodinggradients/ragas/pull/327
feat: improve quality of answer correctness by @shahules786 in https://github.com/explodinggradients/ragas/pull/339
docs: add zeno visualization integration by @cabreraalex in https://github.com/explodinggradients/ragas/pull/359
fix(metrics): avoid getting none in context_recall by @yuukidach in https://github.com/explodinggradients/ragas/pull/355
fix: improved answer relevancy by @shahules786 in https://github.com/explodinggradients/ragas/pull/346
fix(types): fix pyright type issues with latest pyright version by @jjmachan in https://github.com/explodinggradients/ragas/pull/366
Retry for failed json loading by @shahules786 in https://github.com/explodinggradients/ragas/pull/364
fix: handle edge cases in prompt processing by @shahules786 in https://github.com/explodinggradients/ragas/pull/374

New Contributors

@kaiyikang made their first contribution in https://github.com/explodinggradients/ragas/pull/328
@ajinkyavbhandare made their first contribution in https://github.com/explodinggradients/ragas/pull/329
@brunopistone made their first contribution in https://github.com/explodinggradients/ragas/pull/327
@cabreraalex made their first contribution in https://github.com/explodinggradients/ragas/pull/359
@yuukidach made their first contribution in https://github.com/explodinggradients/ragas/pull/355

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.0.21...v0.0.22