Fast (aimed to "real time") Portrait Segmentation on mobile phone
Fast (aimed to "real time") Portrait Segmentation at mobile phone
This project is not normal semantic segmentation but focus on real-time protrait segmentation.All the experimentals works with pytorch.
I hope to find a effcient network which can run on mobile phone. Currently, successfull application of person body/protrait segmentation can be find in APP like SNOW&B612, whose technology is proposed by a Korea company Nalbi.
Encoder : mobilenet_v2(os: 32)
Decoder : unet(concat low level feature) use dilate convolution at different stage(d = 2, 6, 12, 18)
Encoder : shufflenet
Decoder : skip connection (add low level feature)
Attention model is a potential module in the segmentation task. I use a very light residual-dense net as the backbone of the Context Path. The details about fussion of last features in Contxt Path is not clear in the paper(BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation).
Hard segmentation + Soft matting.(coming soon)
:zap: Real-time ! ! ! :tada::tada::tada:
Platform : ncnn.
Mobile phone: Samsung Galaxy S8+(cpu).
model size (M) | time(ms) | |
---|---|---|
model_seg_matting | 3.3 | ~40 |
update : 2018/12/27: Demo video on my iphone 6 (baiduyun)
HUAWEI Mate 20 released recently can keep color on human and make the bacgrand gray in real time (click to view ). I test my model using cpu on my MAC, getting some videos here.