A tool to generate a static website by crawling the original site.
utsusemi = "空蝉"
A tool to generate a static website by crawling the original site.
$ git clone https://github.com/k1LoW/utsusemi.git
$ cd utsusemi
$ npm install
Set environment variables.
OR
Copy config.example.yml
to config.yml
. And edit.
Environment / config.yml Document is here :book: .
$ AWS_PROFILE=XXxxXXX npm run deploy
And get endpoints URL and UtsusemiWebsiteURL
Run following command.
$ AWS_PROFILE=XXxxXXX npm run destroy
/in?path={startPath}&depth={crawlDepth}
Start crawling to targetHost.
$ curl https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/v0/in?path=/&depth=3
And, access UtsusemiWebsiteURL
.
force
optionDisable cache
$ curl https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/v0/in?path=/&depth=3&force=1
/purge
Cancel crawling.
$ curl https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/v0/purge
/delete?prefix={objectPrefix}
Delete S3 object.
$ curl https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/v0/delete?path=/
/status
$ curl https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/v0/status
POST /nin
Start crawling to targetHost with N crawling action.
$ curl -X POST -H "Content-Type: application/json" -d @nin-sample.json https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/v0/nin
depth = depth - 1
depth
.depth = 0
)