Cloudquery Save

Turn any website to API by several clicks (serverless and support SPA!)

Project README

CloudQuery

Join us

Turn any website to API by several clicks.

Try online: https://cloudquery.t9t.io

API

Query params:

  • url: url of the webpage you want to fetch
  • selectors: css selectors describing the elements interests you

Returns:

  • innerText: element innerText
  • href: element href

Run locally

# install dependencies
npm i
# start backend server(for auto restart when code change)
npm run sb
# start frontend server(for auto restart when code change)
npm run sf

# visit http://localhost:1234

How to deploy your own CloudQuery to AWS lambda

1. Configuration

  • update profile(AWS Credential Profiles) in up.json to use your own aws account
  • update rateLimit in config.json to set your own rate limit (Default: 5 request/hour)

2. Deploy

# deploy to aws lambda
up

# see the URL of your CloudQuery
up url

# deploy production version(to cloudquery.t9t.io)
up deploy production

Thanks

  • up for deploying serverless API to AWS with ease
  • serverless-chrome for running chrome on AWS lambda
  • finder for making it easy to select elements on webpage
Open Source Agenda is not affiliated with "Cloudquery" Project. README Source: t9tio/cloudquery
Stars
2,162
Open Issues
31
Last Commit
1 year ago
Repository
License

Open Source Agenda Badge

Open Source Agenda Rating