Towhee

This version is to support both pydantic 1 and 2.

Towhee

This version is to fix the issue of towhee triton server returning incorrect data under high concurrency.

Towhee

This version is to fix incompatibilities with pydantic 2.0.

Towhee

• AutoPipes Support pipelines from Towhee hub

  • towhee/image-embedding
  • towhee/audio-embedding
  • towhee/eqa-insert
  • towhee/eqa-search

• Add debug() API

• Support pipeline as a node

• Add revision and latest for operator

• Add retry when downloading operators

• Add DataSource Operators

  • csv
  • glob
  • sqldb
  • readthedocs

Pipelines

• towhee/eqa-insert
• towhee/eqa-search

Operators

• towhee/text-loader
• towhee/text-splitter
• prompt/question-answer
• prompt/template
• LLM/Dolly
• LLM/OpenAIs
• LLM/Ernie

Document

• docs on official website
• readthedocs

Introduction to New Features of Towhee

• Support batch interface
• Support converting pipeline to Nvidia triton server
• New docs https://docs.towhee.io/ and https://towhee.readthedocs.io/en/main/index.html
• Remove legacy interface.

Towhee.command

• Support operators code package and upload to pypi
  • towhee package -n [namespace] -p [repo_name]
• Support weight files upload to S3, list files in S3 folder and download from S3
  • towhee lsS3 -p [path in bucket]
  • towhee downloadS3 -pl [path local] -pb [path bucket]

s3: model weights

• 5 model weights on S3: clip4clip, vggish, nnfp_fma, nnfp_fma_distill_128

New Models

• 1 Video classification model
  • Vis4mer
• 3 SOTA vision backbones
  • MCProp
  • RepLKNet
  • Shunted Transformer

Operators

• Add 2 code search operators: code_search.codebert, code_search.unixcoder
• Add 5 image captioning operators: image_captioning.expansionnet-v2, image_captioning.magic, image_captioning.clip_caption_reward, image_captioning.blip, image_captioning.clipcap
• Add 5 image-text embedding operators: image_text_embedding.albef, image_text_embedding.ru_clip, image_text_embedding.japanese_clip, image_text_embedding.taiyi, image_text_embedding.slip
• Add 1 machine-translation operator: machine_translation.opus_mt
• Add 1 filter-tiny-segments operator: video-copy-detection.filter-tiny-segments

Notebooks

• Update the beginner tutorial of audio fingerprinting: Audio Fingerprint I: Build a Demo with Towhee & Milvus (modified vote method)
• Add an advanced tutorial for audio fingerprinting: Audio Fingerprint II: Music Detection with Temporal Localization (increased accuracy from 84% to 90%)

Performance

• Improve video de-copy F1 metrics on MPAA dataset.(88.83% -> 94.46%)

Towhee

• New pipeline interface. The pipeline nodes support map/flat_map/filter/window/time_window/window_all types.

New Models:

• Vision backbone for video embedding
  • ISC
• 3 SOTA vision backbones
  • MetaFormer
  • ConvNeXt
  • HorNet

Operators:

• Add 2 video de-copy operators: select-video, temporal-network
• Add 1 image embedding operator specifically designed for image retrieval and video de-copy with SOTA performance on VCSL dataset: isc
• Add 1 audio embedding operator specified for audio fingerprint: audio_embedding.nnfp (with pretrained weights)

Notebooks:

• Add 1 tutorial for video de-copy: video_deduplication_at_segment_level.ipynb
• Add 1 beginner tutorial for audio fingerprint: Audio Fingerprint I: Build a Demo with Towhee & Milvus

DC

• dc.flatten now support multi column
• dc.group_by;
• kv storag mixin
  • dc.insert_leveldb；
  • dc.serach_leveldb;
• Operator output nothing, i.e. dc.operator'input',

towhee

• Use nvidia triton server as a backend of towhee. Towhee supports generating a triton server from a towhee pipeline.
• Towhee supports GPU image decode.

towhee.models

• Add one audio fingerprinting model: nnfp (model & operator)
• Add two image embedding models: RepMLP, WaveViT

5 New SOTA Models:

• Visual recognition, vision-language alignment, image captioning and multimodal understanding with zero-shot transfer, frozen-feature evaluation or end-to-end finetuning.
  • CoCa
• Grounded situation recognition
  • CoFormer
• Repetitive Action Counting
  • TransRAC
• Image retrieval
  • CVNet
• Backbone
  • MaxViT

Bug Fix:

• Reduce towhee packages size.