Giskard Versions Save

🐢 Open-Source Evaluation & Testing framework for LLMs and ML models

v2.12.0

1 week ago

What's Changed

[GSK-1590] native support for claude 3 and titan embeddings on Bedrock by @celmore25 in https://github.com/Giskard-AI/giskard/pull/1905
Fix RAGET correctness value by @pierlj in https://github.com/Giskard-AI/giskard/pull/1910
[GSK-2696] Recover daemonized ML worker with multi ML worker support by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1884
updated mlflow notebook on docs by @rabah-khalek in https://github.com/Giskard-AI/giskard/pull/1912
Documentation for giskard-vision by @bmalezieux in https://github.com/Giskard-AI/giskard/pull/1902
Fixed dependencies installation on macos by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1921
[GSK-2697] Clean up MLWorkerInfo in Python code by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1913
[GSK-3495] Remove eval_prompt parameter from hallucination tests by @henchaves in https://github.com/Giskard-AI/giskard/pull/1916
Update link to doc from latest (main) to stable (last release) by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1922
Bump slackapi/slack-github-action from 1.25.0 to 1.26.0 by @dependabot in https://github.com/Giskard-AI/giskard/pull/1923

New Contributors

@celmore25 made their first contribution in https://github.com/Giskard-AI/giskard/pull/1905

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.11.0...v2.12.0

v2.11.0

3 weeks ago

What's Changed

Polish new README by @alexcombessie in https://github.com/Giskard-AI/giskard/pull/1886
Feature/gsk 2334 talk to my model mvp by @AbSsEnT in https://github.com/Giskard-AI/giskard/pull/1831
Revert "Feature/gsk 2334 talk to my model mvp" by @rabah-khalek in https://github.com/Giskard-AI/giskard/pull/1887
[GSK-3365] Ease custom metrics support by @pierlj in https://github.com/Giskard-AI/giskard/pull/1871
Raget redesign by @davidjanmercado in https://github.com/Giskard-AI/giskard/pull/1895
Fix call to LLM logger breaking scan by @mattbit in https://github.com/Giskard-AI/giskard/pull/1888
Update documentation with support for custom LLM API in generation & evaluation by @TeoAle in https://github.com/Giskard-AI/giskard/pull/1892
Docs update by @pierlj in https://github.com/Giskard-AI/giskard/pull/1898
GSK-3468 Remove Hub documentation from public docs by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1897
formatting fixed by @TeoAle in https://github.com/Giskard-AI/giskard/pull/1899
Added import Giskard and fixed the pip install by @TeoAle in https://github.com/Giskard-AI/giskard/pull/1900
Fix typo by @strickvl in https://github.com/Giskard-AI/giskard/pull/1890
Fixed coherency test key by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1891
Add detector param for min slice size by @luca-rossi in https://github.com/Giskard-AI/giskard/pull/1822
Fix sonarcloud issues by @pierlj in https://github.com/Giskard-AI/giskard/pull/1896
Feature/gsk 2334 talk to my model mvp by @AbSsEnT in https://github.com/Giskard-AI/giskard/pull/1889
Fix raget docs by @pierlj in https://github.com/Giskard-AI/giskard/pull/1904

New Contributors

@davidjanmercado made their first contribution in https://github.com/Giskard-AI/giskard/pull/1895
@strickvl made their first contribution in https://github.com/Giskard-AI/giskard/pull/1890

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.10.0...v2.11.0

v2.10.0

1 month ago

What's Changed

Remove uppercas F in examples, so model return proper JSON by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1877
Readme updates by @luca-martial in https://github.com/Giskard-AI/giskard/pull/1878
Full Apache 2 license by @alexcombessie in https://github.com/Giskard-AI/giskard/pull/1885
Support for custom LLM clients by @mattbit in https://github.com/Giskard-AI/giskard/pull/1839
Bump softprops/action-gh-release from 1 to 2 by @dependabot in https://github.com/Giskard-AI/giskard/pull/1873
[GSK-3367] Adding brier score + monotonicity + smoothness tests by @bmalezieux in https://github.com/Giskard-AI/giskard/pull/1863

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.9.1...v2.10.0

v2.9.1

1 month ago

What's Changed

[GSK-2879][GSK-3300] Technical documentation of Giskard Evaluator in the Giskard doc by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1848
New tools image by @TeoAle in https://github.com/Giskard-AI/giskard/pull/1882
GIFs update by @TeoAle in https://github.com/Giskard-AI/giskard/pull/1883

New Contributors

@TeoAle made their first contribution in https://github.com/Giskard-AI/giskard/pull/1882

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.9.0...v2.9.1

v2.9.0

1 month ago

What's Changed

[GSK-2497] Allow to connect multiple external mlworkers by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1740
Switching from mlWorkerId to kernel name by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1777
GSK-2896 Automatically generate a Kernel based on dependencies by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1829
GSK-3241 Make kernel_name the last argument of create_project by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1838
GSK-2887 Upload tests with project when uploading a suite by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1837
GSK-2928 GSK-2877 Added a method to start and stop managed ML worker using the … by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1851
Update lockfile & clear runner disk by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1868
Update RAGET docs by @pierlj in https://github.com/Giskard-AI/giskard/pull/1874
[GSK-3419] Add mixpanel telemetry to new client methods by @henchaves in https://github.com/Giskard-AI/giskard/pull/1875
[GSK-3430] Check if we can detect the version of giskard library by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1876
[GSK-3442] Install giskard only on google colab by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1879
[GSK-2877] Update doc about ML worker to adapt kernel and multi-ML worker by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1853
[GSK-2497] Allow to connect multiple external mlworkers (#1740) by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1771
Small documentation updates with latest product story by @alexcombessie in https://github.com/Giskard-AI/giskard/pull/1880
RAG Toolkit: Out of KB questions [GSK-3372] by @JiaenLiu in https://github.com/Giskard-AI/giskard/pull/1861

New Contributors

@JiaenLiu made their first contribution in https://github.com/Giskard-AI/giskard/pull/1861

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.8.0...v2.9.0

v2.8.0

1 month ago

What's Changed

[GSK-2892] Improve RAG toolset by @pierlj in https://github.com/Giskard-AI/giskard/pull/1820
[GSK-3361] Model predictions are casted to double by default in regression mode by @bmalezieux in https://github.com/Giskard-AI/giskard/pull/1857
Update mlflow-llm-example.ipynb (typo) by @rabah-khalek in https://github.com/Giskard-AI/giskard/pull/1858

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.7.7...v2.8.0

v2.7.7

1 month ago

What's Changed

[GSK-3303] Fix validation issue with tuple, also fixes param names by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1844
Fix the llm client to reset when changing llm api or model by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1842
Issue with evaluating openai-based LLMs with mlflow.evaluate() API by @rabah-khalek in https://github.com/Giskard-AI/giskard/pull/1843
GSK-2409 Add doc to how to update the lockfile by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1840
hot-fixing transformation in push perturbation by @rabah-khalek in https://github.com/Giskard-AI/giskard/pull/1849
Prevent torch faulty version on MacOS by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1855

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.7.6...v2.7.7

v2.7.6

2 months ago

What's Changed

[GSK-2937] Update docs after moving non-ML worker code outside by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1836
GSK-2604 GSK-2766 Renamed tittle of wage classification notebook by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1841

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.7.5...v2.7.6

v2.7.5

2 months ago

What's Changed

[GSK-2920] Fallback to english if num2words does not implement in a language by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1826
Fix LLMON url typo by @alexcombessie in https://github.com/Giskard-AI/giskard/pull/1828
Bump pre-commit/action from 3.0.0 to 3.0.1 by @dependabot in https://github.com/Giskard-AI/giskard/pull/1827
GSK-1807 Integrate Sentry by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1823
Fix download link for enron by @Hartorn in https://github.com/Giskard-AI/giskard/pull/1834
GSK-2902 GSK-2418 Add a method to execute test suite and upload result to Hub programmatically + result cleanup by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1818

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.7.4...v2.7.5

v2.7.4

2 months ago

What's Changed

[GSK-2871] Fix TextPerturbationDetector failed with error from num2word under "it" or "fa" language code by @Inokinoki in https://github.com/Giskard-AI/giskard/pull/1815
GSK-2874 Fix quality badge using https://peps.python.org/pep-0563/ by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1816
[GSK-2793] Updated injection prompts (removing duplicates) by @rabah-khalek in https://github.com/Giskard-AI/giskard/pull/1817
Remove verbose log when wrapping dataset by @luca-rossi in https://github.com/Giskard-AI/giskard/pull/1821
Bump docs date to 2024 by @luca-martial in https://github.com/Giskard-AI/giskard/pull/1819
GSK-2376 Add seed to LLM generators and evaluators in order to ensure reproducibility by @kevinmessiaen in https://github.com/Giskard-AI/giskard/pull/1824

Full Changelog: https://github.com/Giskard-AI/giskard/compare/v2.7.3...v2.7.4