Cloudproxy Versions Save

Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.

v0.4.0-beta

2 years ago
  • #24 Bugfix AWS delete only checking the first instance
  • Change retries to 1 and set a timeout of 10s on fetch_ip
  • Rewrote the check_alive function to be much simpler, the fetch_ip check was not viable at 20+ proxies. It took too long.
  • Updated ip_list not to return IP:s slated for destruction
  • Added option to restart AWS proxies, much faster than destroy/create and fetches a new IP. Not supported for DO.
  • Opened up port 22 on proxies for debugging, future enhancement is to only allow web control. (Use the EC2_INSTANCE_CONNECT filter for the service parameter to get the IP address ranges in the EC2 Instance Connect subset. )
  • Enhanced status messages a bit in the check_alive for AWS
  • Moved check_delete/stop to before provision so it removes and then immediately provisions a new one instead of waiting 20s for the next tick
  • Changed the proxy software to tinyproxy directly on the image instead of using docker. Much faster deployment and less CPU intensive so should work better with t2.nano
  • Updated the settings checks to compare true/false as a string since it seems to be what it is getting, earlier a value of False in the config would read as true.
  • Updated environ get to match the doc (ie SCALING instead of SCALE)
  • Added botocore to requirements.txt, which seemed to be missing.

@xanrag thank you for all these fixes.

v0.3.3-beta

2 years ago
  • Fixed AWS provisioning issue.

v0.3.2-beta

2 years ago
  • Age limit feature added
    • Set in seconds how long the proxy should live for, when the limit is hit, the proxy and IP is replaced.

v0.3.1-beta

2 years ago
  • Fixed issue with API endpoints in UI:
    • If CloudProxy was running on any other port than 8000, the UI would not function, now fixed

v0.3.0-beta

3 years ago

Added CloudProxy UI!

v0.2.2-beta

3 years ago
  • Updated error handling
  • Added retry to check alive
  • Added CORS and delete_queue now set
  • Schedule with providers every 20 seconds now and removed auth from IP
  • Fixed failing tests

v0.2.1-beta

3 years ago

Added support for AWS as a provider.

v0.2.0-beta

3 years ago

New features:

  • Manage providers via the API
  • Additional configuration options for DigitalOcean provider
  • Improved test coverage

v0.1.1-alpha

3 years ago

Improving setting of auth creds and readme

v0.1.0-alpha

3 years ago