NEMOSIS Versions Save

• Add new table, NEXT_DAY_DISPATCHLOAD, which contains the same data as DISPATCHLOAD but is available the next day. Data source is here: https://nemweb.com.au/Reports/Current/Next_Day_Dispatch/.
• Documentation for NEXT_DAY_DISPATCHLOAD and DAILY_REGION_SUMMARY still need to be added to the NEMOSIS wiki.

• Change the source of bidding data so that bids are available next day, this also means there should no gap in the bidding data accessed through NEMOSIS. The new data source is here: http://www.nemweb.com.au/REPORTS/CURRENT/Bidmove_Complete/
• Add access to regional summary data published the next day. The data source is here: http://www.nemweb.com.au/REPORTS/CURRENT/Daily_Reports/.

• Replace print messages with logging, this means messages displayed by NEMOSIS can be controlled by the user as per the readme documentation. Thanks to @prakaa
• Fix bug where attempting to download bidding data that is not available on nemweb would cause NEMOSIS to crash. Not AEMO stopped uploading monthly bidding files to the public archive as of April 2021. If NEMOSIS can't find bidding in the archive, it attempts to download from the 'bid move complete' dataset, which only hosts a 12 month window of the most recent bidding data (https://www.nemweb.com.au/REPORTS/Archive/Bidmove_Complete/). If the bidding data is not available a warning is logged.

• Fixed #20 where NEMOSIS couldn't access bid data after March 2021 because AEMO changed the format in which the data was published. Thanks @dawkinsmw for helping put together the methodology for extracting the data from the new format. Note with the format change there is still a window of missing data which includes March 2021, the 1st of April 2021, and the first 4 hrs of the 2nd of April.
• Added the rebuild option to the dynamic_data_compiler function, this forces NEMOSIS to rebuild feather or parquet cache files from the raw CSV data. This is useful if you want to add extra columns to the data cache, as described here: https://github.com/UNSW-CEEM/NEMOSIS#accessing-additional-table-columns

Major version increment due to small but breaking changes to the API

• removed data_merge argument from dynamic data_compiler as cache_compiler now contains this functionality
• invalid arguments to data_compiler and cache_compiler now raise errors
• all static tables can now be accessed using the static_table function. Functions static_table_FCAS_elements_file and static_table_xl have been removed.

Major version increment: Updated API, thanks to @prakaa for #15. New features are: 1) type parsing for data returned via the API, 2) cache_compiler function that builds a cache of typed feather or parquet files. See the updated readme for more details.

Patch version increment: Removed columns SEMISCHEDULE_CLEAREDMW, SEMISCHEDULE_COMPLIANCEMW from TRADINGLOAD defaults

Updated API. Thanks to @prakaa for #11

Fixed column bug. Thanks to @jotaigna #12

OS independent path concatenation gui creates directories if they don't exist already

Thanks to @jurasofish for the updates.

Minor changes to accommodate AEMO's website update.

Added new merging type based off pandas merge as of. Lets you merge the left time column with the most recent record in the right time column. For more details read: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.merge_asof.html

Also added filter to select for highest numbered version of a given record. Data is grouped by the remaining primary key columns and the record with highest version number is kept. Should only be used on single AEMO tables that have not been merged with other AEMO tables.