BentoML Versions Save

The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!

v1.2.12

1 week ago

What's Changed

docs: Add GPU inference doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4654
chore: update quickstart by @ssheng in https://github.com/bentoml/BentoML/pull/4655
docs: Add JSON output for bentovllm by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4657
chore: cleanup quickstart by @ssheng in https://github.com/bentoml/BentoML/pull/4658
docs: Update help info by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4664
fix: remove the uvicorn server header by @frostming in https://github.com/bentoml/BentoML/pull/4665
docs: Fix format by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4666
docs: Add model composition doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4668
docs: Update example project list by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4673
docs: Add the monitoring and data collection doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4662
docs: Add add_asgi_middleware doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4672
fix: delete useless enum and fix enum value by @FogDong in https://github.com/bentoml/BentoML/pull/4674
docs: Add RAG tutorial by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4675
docs: Update the clients doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4676
docs: Add some explanations for bentoml.models.get by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4660
docs: Add e2e test doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4679
fix(cloud client): various type error by @bojiang in https://github.com/bentoml/BentoML/pull/4680
fix(cli): bentoml cli verbosity not passed to the subprocess correctly by @frostming in https://github.com/bentoml/BentoML/pull/4661

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.11...v1.2.12

v1.2.11

2 weeks ago

What's Changed

docs: added multi model load and store example by @lycheel1 in https://github.com/bentoml/BentoML/pull/4575
docs: inference graph by @lycheel1 in https://github.com/bentoml/BentoML/pull/4609
docs: Update high-level BentoML messaging by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4644
fix: delete trace error for timeout when deploying by @FogDong in https://github.com/bentoml/BentoML/pull/4649
fix(cli): correct the comparison of upload status by @frostming in https://github.com/bentoml/BentoML/pull/4643

New Contributors

@lycheel1 made their first contribution in https://github.com/bentoml/BentoML/pull/4575

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.10...v1.2.11

v1.2.10

3 weeks ago

What's Changed

docs: Remove old example links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4610
feat(build): use jinja template for install script by @aarnphm in https://github.com/bentoml/BentoML/pull/4607
feat: download git+ssh dependencies to wheels/ folder by @frostming in https://github.com/bentoml/BentoML/pull/4612
docs: Update the ASGI doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4614
fix: non batched request executed in non FIFO manner by @frostming in https://github.com/bentoml/BentoML/pull/4619
feat: light-weighted dev sdist builder by @frostming in https://github.com/bentoml/BentoML/pull/4615
fix: add response headers for error responses by @frostming in https://github.com/bentoml/BentoML/pull/4622
docs: Refactor the autoscaling doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4620
docs: Add simple verification test for Services by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4629
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4630
docs: Add workers doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4628
docs: Add worker doc links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4636
docs: Add docs for bentoml.runner_service by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4637
docs: Add sync to async doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4639
fix: handle call on clients by @frostming in https://github.com/bentoml/BentoML/pull/4618
fix(cli): bentoml get output in YAML format by @frostming in https://github.com/bentoml/BentoML/pull/4638
fix: issue overriding default service config from config file by @frostming in https://github.com/bentoml/BentoML/pull/4627

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.9...v1.2.10

v1.2.9

1 month ago

What's Changed

docs: add autoscaling docs by @ssheng in https://github.com/bentoml/BentoML/pull/4601
docs: Fix old configurations doc link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4600
docs: Update the concurrency doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4602
fix(reload): make sure to watch for all src for editable by @aarnphm in https://github.com/bentoml/BentoML/pull/4604
fix(openapi): remove invalid fields for operations parameter by @aarnphm in https://github.com/bentoml/BentoML/pull/4605
chore: update swagger UI generated to 5.12 [skip ci] by @aarnphm in https://github.com/bentoml/BentoML/pull/4606
fix(cloud cli): workaround env update issue by @bojiang in https://github.com/bentoml/BentoML/pull/4608

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.8...v1.2.9

v1.2.8

1 month ago

What's Changed

fix(stub): add envs to __init__ construction by @aarnphm in https://github.com/bentoml/BentoML/pull/4576
docs: Add testing doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4578
chore(type): use TypedDict for inferred stubs by @aarnphm in https://github.com/bentoml/BentoML/pull/4577
docs: Add lifecycle hooks doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4579
fix(build): remove deprecated warnings by @aarnphm in https://github.com/bentoml/BentoML/pull/4580
fix: improve the bentoml build progress by @frostming in https://github.com/bentoml/BentoML/pull/4582
fix: allow API endpoint to return response directly by @frostming in https://github.com/bentoml/BentoML/pull/4581
fix: raise the priority of the routes from mount apps by @frostming in https://github.com/bentoml/BentoML/pull/4584
fix(sdk): python build missing envs argument by @jianshen92 in https://github.com/bentoml/BentoML/pull/4585
fix: build config env default to None by @jianshen92 in https://github.com/bentoml/BentoML/pull/4589
fix: config from multiple services override with each other by @frostming in https://github.com/bentoml/BentoML/pull/4591
feat: add concurrency config by @xianml in https://github.com/bentoml/BentoML/pull/4560
docs: Add OpenAI compatible endpoints client doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4593
fix: ignore_extra_keys for v2 configuration schema by @xianml in https://github.com/bentoml/BentoML/pull/4598
fix: emit warnings when copying large file by @frostming in https://github.com/bentoml/BentoML/pull/4597
feat: service.to_async property by @frostming in https://github.com/bentoml/BentoML/pull/4595
feat: picklev5 serialization by @frostming in https://github.com/bentoml/BentoML/pull/4594

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.7...v1.2.8

v1.2.7

1 month ago

What's Changed

docs: Add doc for server_ready_timeout by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4563
fix: depracate all enum in schema by @FogDong in https://github.com/bentoml/BentoML/pull/4565
feat: threaded serving by @frostming in https://github.com/bentoml/BentoML/pull/4562
docs: Adjust existing Bento deployment wording by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4567
fix: fix list instance type by @FogDong in https://github.com/bentoml/BentoML/pull/4566
docs: Add doc for using pydantic model at the top level by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4568
fix: delete access in admin console by @FogDong in https://github.com/bentoml/BentoML/pull/4569
fix(client): set upload timeouts to 10 hours by @sauyon in https://github.com/bentoml/BentoML/pull/4564
fix: respect the content type config for file output by @frostming in https://github.com/bentoml/BentoML/pull/4570
docs: Update the adaptive batching doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4571
feat: Service.to_asgi() method by @frostming in https://github.com/bentoml/BentoML/pull/4572
docs: Add asgi doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4574
fix: allow legacy runnable methods when calling API by @frostming in https://github.com/bentoml/BentoML/pull/4573

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.6...v1.2.7

v1.2.6

1 month ago

What's Changed

docs: Add envs field in bentofile by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4555
docs: Add GitHub repo links for use case docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4556
docs: Update the vllm use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4558
docs: Add how to get deployment info doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4544
fix: rename the _bentoml_sdk.api module by @frostming in https://github.com/bentoml/BentoML/pull/4557
fix: wait until server ready when initializing client by @frostming in https://github.com/bentoml/BentoML/pull/4559
feat: Add server.on_deployment hook to execute deployment-specific logic by @frostming in https://github.com/bentoml/BentoML/pull/4561

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.5...v1.2.6

v1.2.5

1 month ago

What's Changed

docs: Add client code examples without context manager by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4512
docs: Update docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4515
docs: Add authorization docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4517
docs: Change sample input to one line by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4518
docs: Update ControlNet use case docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4519
docs: Update the distributed services and get started docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4521
refactor(cli): make CLI commands available as modules by @frostming in https://github.com/bentoml/BentoML/pull/4487
docs: Refactor BentoCloud docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4525
docs: Update token docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4526
fix: fix list deployment exception by @FogDong in https://github.com/bentoml/BentoML/pull/4522
feat: add get endpoint url for deployment by @FogDong in https://github.com/bentoml/BentoML/pull/4516
docs: Add BentoCloud CLI and API ref docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4527
fix: add missing to dict for deployment config by @FogDong in https://github.com/bentoml/BentoML/pull/4529
feat(client): disallow media type other than json for HTTP clients by @frostming in https://github.com/bentoml/BentoML/pull/4520
chore: clean unneeded dependencies by @frostming in https://github.com/bentoml/BentoML/pull/4528
docs: Add call deployment endpoints doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4531
docs: Update all use cases by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4532
fix: correctly encode ValueError in response by @frostming in https://github.com/bentoml/BentoML/pull/4530
fix: improve CLI message by @frostming in https://github.com/bentoml/BentoML/pull/4533
chore(io): allow implicit pydantic type conversion by @aarnphm in https://github.com/bentoml/BentoML/pull/4534
fix: type convertion of io descriptor by @frostming in https://github.com/bentoml/BentoML/pull/4537
feat: add back service hooks by @frostming in https://github.com/bentoml/BentoML/pull/4524
docs: fix typo by @sauyon in https://github.com/bentoml/BentoML/pull/4540
docs: Add BentoSVD Use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4535
doc: Add adaptive batching doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4536
doc: allow reuse of existing directory by @jianshen92 in https://github.com/bentoml/BentoML/pull/4543
feat: cloud context from env vars by @frostming in https://github.com/bentoml/BentoML/pull/4542
docs: Update the clients doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4539
docs: Update adaptive batching docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4547
docs: Update vLLM use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4548
feat: export component context at top namespace by @frostming in https://github.com/bentoml/BentoML/pull/4551
feat: use starlette request directly in service context by @frostming in https://github.com/bentoml/BentoML/pull/4552
feat: mount asgi app and fastapi support by @frostming in https://github.com/bentoml/BentoML/pull/4538
fix(sdk): build envs into dockerfile by @bojiang in https://github.com/bentoml/BentoML/pull/4546
fix: rename server context attributes by @frostming in https://github.com/bentoml/BentoML/pull/4553
fix: crash when pulling bento from organization with proxy mode by @frostming in https://github.com/bentoml/BentoML/pull/4554
fix: add str in enum for deployment schema by @FogDong in https://github.com/bentoml/BentoML/pull/4549

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.4...v1.2.5

v1.2.4

2 months ago

What's Changed

docs: Update the get started docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4513
fix(sdk): incorrect bento_path if not provided by @bojiang in https://github.com/bentoml/BentoML/pull/4514

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.3...v1.2.4

v1.2.3

2 months ago

What's Changed

ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4488
infra: update to use Ruff formatter by @aarnphm in https://github.com/bentoml/BentoML/pull/4269
feat: tpu_type support by @frostming in https://github.com/bentoml/BentoML/pull/4493
chore(cloud cli): add cluster and expose access_point by @bojiang in https://github.com/bentoml/BentoML/pull/4490
docs: Update the whisperx use case by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4495
docs: Update the readme by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4496
fix(sdk): current directory for built bentos by @bojiang in https://github.com/bentoml/BentoML/pull/4505
chore(cloud cli): rename cluster to region by @bojiang in https://github.com/bentoml/BentoML/pull/4508
doc: Add the lcm lora use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4510
fix(sdk): clean bentoml version by @bojiang in https://github.com/bentoml/BentoML/pull/4511
fix: bug: Dataframes not serializing correctly in the new API by @frostming in https://github.com/bentoml/BentoML/pull/4491

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.2...v1.2.3