Official implementation of the ICASSP-2022 paper "Text2Poster: Laying Out Stylized Texts on Retrieved Images"
The inference code of the ICASSP-2022 paper "Text2Poster: Laying Out Stylized Texts on Retrieved Images".
Paper Link: https://arxiv.org/abs/2301.02363
import time, json, requests
timestamp = time.strftime('%Y%m%d%H%M%S',time.localtime(time.time()))
input_text_elements = {
"sentences": [
["CHILDREN'S DAY", 90], # [text, font_size]
["Children are The Future of Nation", 50] # [text, font_size]
],
"background_query": "Children's Day!" # sentence used to retrieve background images.
}
input_text_elements = json.dumps(input_text_elements)
api_url = "http://bl.mmd.ac.cn:8889/text2poster"
response = requests.get(api_url, params = {"input_text_elements": input_text_elements})
if response.status_code == 200:
f = open("poster-{}.jpg".format(timestamp), "wb")
f.write(response.content)
f.close()
print("Save poster to:", "poster-{}.jpg".format(timestamp))
else:
print(response.text)
./background_retriever/unsplash_image_downloader.py
, you can use this script to get those background image files based on the retrieved image urls../background_retriever
for the convenience of people who are not in mainland China to use our Text2Poster. You can retrieve the background image locally, which requires about 3GB of GPU memory.
55, 40 and 30 are the font size.
{
"sentences": [
["冬日初雪舞会", 55],
["雪花飞舞,像音乐和歌声围绕", 40],
["与朋友相聚,享受欢乐时光,我们不见不散", 30]
],
"background_query": "冬日初雪舞会"
}
80 and 55 are the font size.
{
"sentences": [
["ICASSP 2022", 80],
["May 22 - 27, 2022, Singapore", 55]
],
"background_query": "Singapore"
}
output posters
90 and 50 are the font size.
{
"sentences": [
["桜が咲く", 90],
["出会いは素晴らしい春に", 50]
],
"background_query": "春の美しい桜"
}
output posters
We recommend you use anaconda to run our Text2Poster. Run the following command to install the dependent libraries:
bash install_package.sh
you also can install the dependent libraries manually:
# using the tsinghua mirror to speed up the install.
conda install pytorch=1.10.0 -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
conda install torchvision=0.11.0 -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
pip install opencv_contrib_python
pip install transformers==3.2.0
pip install argparse
pip install freetype-py
pip install requests
pip install jsonlines
pip install tqdm
pip install pyyaml
pip install easydict
pip install timm
We provide the following resource to start Text2Poster:
./checkpoint/0.20484_Cascading_128_uniform_big.pth
;./checkpoint/27.80619_distribCNN_BigPosition_epoch_76_scale_20.pth
;[Not required] Although we provide an API for background image retrieval, if you want to retrieve background images from the source code, you need to download the following resources:
./background_retriever/weights/
;./background_retriever/background_feats/
;./background_retriever/background_feats/unsplash_image_url.jsonl
.We provide two example, Run the following command to run our Text2Poster:
bash run.sh
Some parameters:
python background_retrieval.py
cd background_retriever
python main.py
python layout_distribution_predict.py
python layout_refine.py
python ./background_retriever/unsplash_image_downloader.py
we also output some intermediate processing files in ./example/outputs
:
Blue region: The saliency map;
Green region: The predicted layout distribution map;
Red region: the predicted layout map.
python==3.7
pytorch=1.10.0
torchvision=0.11.0
transformers==3.2.0
freetype-py
opencv_contrib_python
requests
jsonlines
tqdm
argparse
pyyaml
easydict
timm
If you find this paper and repo useful, please cite us in your work:
@inproceedings{DBLP:conf/icassp/JinXSL22,
author = {Chuhao Jin and
Hongteng Xu and
Ruihua Song and
Zhiwu Lu},
title = {Text2Poster: Laying Out Stylized Texts on Retrieved Images},
booktitle = {{IEEE} International Conference on Acoustics, Speech and Signal Processing,
{ICASSP} 2022, Virtual and Singapore, 23-27 May 2022},
pages = {4823--4827},
publisher = {{IEEE}},
year = {2022},
url = {https://doi.org/10.1109/ICASSP43922.2022.9747465},
doi = {10.1109/ICASSP43922.2022.9747465},
timestamp = {Tue, 07 Jun 2022 17:34:56 +0200},
biburl = {https://dblp.org/rec/conf/icassp/JinXSL22.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
My Email is: [email protected]