JobFunnel Versions Save

Scrape job websites into a single spreadsheet with no duplicates.

3.0.2

3 years ago

This release addresses a number of things, including one critical encoding bug:

  • Removes brotli compression from request HTML (this recently caused all of our scrapers to break).
  • Reduction in Error log messages
  • PyPi compatibility
  • Switched to Github Actions from TravisCI (no longer free)
  • Functional Proxy support
  • Improved error handling

Thank you to our contributors!

@thebigG @marchbnr @markkvdb

3.0.1

3 years ago

This release adds some new features:

  • you can now search for jobs by 'remoteness' i.e. TEMPORARILY_REMOTE, FULLY_REMOTE
  • improved support for wage scraping
  • new locales: UK_ENGLISH and FRANCE_FRENCH
  • improved scraping logic

Thanks to our contributors! @LilySu @markkvdb

3.0.0

3 years ago

This release brings some significant improvements to the JobFunnel codebase:

  • Support for localization, for now just USA_ENGLISH CANADA_ENGLISH
  • Abstract base class implementation of JobFunnel and Scrapers with abstract get() and set() style of API
  • Addition of remote and wage scraping (to be implemented for indeed and monster)
  • Updated scraping for Monster and Glassdoor
  • Improved CLI argument clarity (load and inline)
  • Implementation of Cerberus for Schema and validation of YAML configuration files
  • Significantly improved preemption of scraping and filtering of results to minimize get requests
  • Implementation of inter-scraper OO features such as Job JobField and JobFilter
  • Capability of updating CSV job contents when encountering a newer duplicate

... and others.

Thanks to our contributors :100: @bunsenmurder @thebigG @studentbrad

2.1.8

3 years ago

In this release:

  • addition of a webdriver
  • significantly increased code coverage
  • implementation of a dynamic scraper for GlassDoor (try it out by enabling GlassDoorDynamic in your settings.yaml)

Thanks to our contributors for this release! @thebigG @studentbrad @Arax1

2.1.5

4 years ago

The tests folder was moved outside of the JobFunnel package. It is no longer included in the installation.

Additional changes by @thebigG and @studentbrad include:

  • More code coverage
  • Coverage reports

2.1.4

4 years ago

New date filter made by @thebigG as well as improved CLI and log path fixes.

  • Added a date filter so that you can now prevent adding jobs to the masterlist which are older than a configurable max_listing_days
  • Added additional options for CLI (i.e. funnel -kw Java --city 'New York' -p NY --domain com)
  • Added Windows support in log path calculations

2.1.1

4 years ago

Fixes the pip installation by including package data.

2.1.0

4 years ago

User enhancements made by @markkvdb including:

  • configuration validation
  • proxy settings

Unit tests for better code coverage made by @studentbrad and @markkvdb including:

  • test_countries.py
  • test_parse.py
  • test_tools.py
  • test_validate.py

For more information on unit tests see #55.

2.0.0

4 years ago

Major stability and performance improvements from @bunsenmurder including:

  • respectful (delay-able) scraping
  • scrapes more data from indeed
  • tfidf accuracy improvement
  • refactoring for clarity & PEP8
  • architectural improvements
  • improved threading

1.1.3

4 years ago