Pmaw Versions Save

A multithread Pushshift.io API Wrapper for reddit.com comment and submission searches.

v1.0.2

3 years ago

Bugfix

  • fixed comments returning 25 by default

v1.0.1

3 years ago

Bugfix

  • threw error when limit was undefined

v1.0.0

3 years ago

New Features / Improvements

  • search methods now return a Response generator object
  • memory safety can now be enabled with mem_safe to cache responses during data retrieval and reduce the amount of memory used
  • safe exiting can now be enabled with safe_exit to safely exit when an interrupt signal is received during data retrieval
  • load unfinished requests and saved responses from cache when safe exiting is enabled
  • request details are now handled inside a Request object

v0.1.3

3 years ago

Bugfixes

  • Fixed infinite while loop error
  • Checkpoint by batch
  • Removed erroneous pandas import

v0.1.2

3 years ago

Bugfixes

  • Fixed time slicing creating extra requests which were less than the max_results_per_request

v0.1.1

3 years ago

Bugfixes

  • Fixed a bug with time slicing causing duplicate results
  • Fixed a miscalculation error for remaining results for a timeslice

Other

  • Initial check for total results for provided parameters updated to use multithreading

v0.1.0

3 years ago

New Features / Improvements

  • General code improvements
  • Added more rate-limiting options with exponential backoff with different types of jitter
  • Added non-id based search for submissions and comments

Other

  • Benchmarked PMAW rate-limiting strategies against PSAW with default parameters

v0.0.2

3 years ago

New Features / Improvements


  • Initial implementation of multithreading requests for ids queries, with support for:
    • comment ids by submission id
    • submissions by id
    • comments by id
  • Rate-limit based on rate averaging across previous requests