cot_reports is a Python library for fetching the Commitments of Trader reports of the Commodity Futures Trading Commission (CFTC). The following COT reports are supported: Legacy Futures-only, Legacy Futures-and-Options Combined, Supplemental Futures-and-Options Combined, Disaggregated Futures-only, Disaggregated Futures-and-Options Combined, Traders in Financial Futures (TFF) Futures-only and Traders in Financial Futures (TFF) Futures-and-Options Combined.
cot_reports is a Python library for fetching the Commitments of Trader reports of the Commodity Futures Trading Commission (CFTC). The following Commitments of Trader reports are supported: Legacy Futures-only, Legacy Futures-and-Options Combined, Supplemental Futures-and-Options Combined, Disaggregated Futures-only, Disaggregated Futures-and-Options Combined, Traders in Financial Futures (TFF) Futures-only and Traders in Financial Futures (TFF) Futures-and-Options Combined.
Please note: The functionality of the libraries' functions may interrupt in case any changes in the publication occur or in case the website is not available.
Use the package manager pip to install cot_reports
pip install cot_reports
Or install the library through the author's Github repository
pip install git+https://github.com/NDelventhal/cot_reports
The following libraries are required:
These libraries can be installed via the package manager pip.
pip install pandas requests beautifulsoup4
Available report types with the corresponding 'cot_report_type' name used in the library:
report | 'cot_report_type' |
---|---|
Legacy Report Futures-only | 'legacy_fut' |
Legacy Report Futures-and-Options Combined | 'legacy_futopt' |
Supplemental Report Futures-and-Options combined (short format) | 'supplemental_futopt' |
Disaggregated Report Futures-only | 'disaggregated_fut' |
Disaggregated Report Futures-and-Options Combined | 'disaggregated_futopt' |
Traders in Financial Futures (TFF) | 'traders_in_financial_futures_fut' |
Traders in Financial Futures (TFF) Futures-and-Options Combined | 'traders_in_financial_futures_futopt' |
Usage examples:
cot_hist() downloads the compressed bulk files for the specified report (cot_report_type) starting from 1986 (depending on the report type) up to 2016 and returns the data in a dataframe.
cot_year() downloads historical single year data for the specified report (cot_report_type) and returns the data in a dataframe (cot_report_type). If the current year is specified, the latest published data is fetched.
cot_all() downloads the complete available data, including the latest, of the specified report and returns the data in a dataframe (cot_report_type).
cot_all_reports() downloads all available historical information of the available COT reports and returns the data as seven dataframes.
import cot_reports as cot
# Example: cot_hist()
df = cot.cot_hist(cot_report_type= 'traders_in_financial_futures_futopt')
# cot_hist() downloads the historical bulk file for the specified report type, in this example the Traders in Financial Futures Futures-and-Options Combined report. Returns the data as dataframe.
# Example: cot_year()
df = cot.cot_year(year = 2020, cot_report_type = 'traders_in_financial_futures_fut')
# cot_year() downloads the single year file of the specified report type and year. Returns the data as dataframe.
# Example for collecting data of a few years, here from 2017 to 2020, of a specified report:
df = pd.DataFrame()
begin_year = 2017
end_year = 2020
for i in range(begin_year, end_year + 1):
single_year = pd.DataFrame(cot.cot_year(i, cot_report_type='legacy_futopt'))
df = df.append(single_year, ignore_index=True)
# Example: cot_all()
df = cot.cot_all(cot_report_type='legacy_fut')
# cot_all() downloads the historical bulk file and all remaining single year files of the specified report type. Returns the data as dataframe.
To promote its goals of integrity, resilience and vibrancy of the U.S. derivatives markets through regulation, the U.S. Commodity Futures Trading Commission (CFTC) relies on collected data to conduct its functions. The CFTC's functions:
In order to conduct these functions, data is seen as a critical asset to the CFTC, which states to have been one of the first federal agencies who implemented a data management organization in 2011. The CFTC is not solely collecting but also publishing a variety of reports on the derivative markets under its jurisdiction. These reports include the Commitment of Traders (COT) reports.
Through the publication of the COT reports the CFTC claims 'to help the public to understand market dynamics.'
The COT reports provide a breakdown of each Tuesday’s open interest for futures and options on futures markets in which 20 or more traders hold positions equal to or above the reporting levels established by the CFTC.
The data is reported to the CFTC by firms, such as:
The actual trader classification is based on the predominant business purpose, which is self-reported by traders on the CFTC Form 401. The classification of a trader can vary for different commodities/markets, if the reported business purposes differ. However, the classifications are subject to review and corrections by the CFTC.
The data, which is generally released each Friday at 3:30 pm Eastern time, comes with a lag of three days, as the reported data is typically from previous Tuesday (close). According to the commission the firms report to the CFTC on Wednesday morning. Next, the provided information is then subject to corrections and verifications by the CFTC prior to the release on Friday.
The types of the COT reports:
Legacy
Supplemental
Disaggregated
Traders in Financial Futures
The COT reports are either available in the short format, the long format or both - as specified above. Within the long format, aside from the additional listing of the concentrations of positions held by the largest four and largest eight traders, the data is grouped by crop year, where appropriate. Except, where not available (cot report type: Supplemental), the library accesses the long format.
Aggregations occur in the reports not only in the classification of the trader, but also by merging future positions with differing expiration dates. Due to this handling continuous sentiment data on a product, referred to as market by the CFTC and within the COT reports, can be derived. As the data is highly aggregated, any results derived from the data should be interpreted with caution.
Not only does the Legacy Futures-only COT report offer the greatest available historical data dating back to 1986, the Legacy reports also contain the most markets in comparison to the other COT report types. Unlike the others, the Legacy reports contain data on physical delivery contracts and non-physical delivery (financial) contracts. While the Supplemental and the Disaggregated reports are limited to physical delivery contracts, the Traders in Financial Futures reports are limited to non-physical financial contracts.
Classification Methodology: Legacy reports
The positions of the market participants in the Legacy reports are broken down into:
Commercials
Non-commercials
Non-reportable
There are numerous, partially overlapping, use cases for the COT data of the CFTC. Here are the ones I can think of:
Regulatory/Academic:
Central banks:
Market participants:
Market observers:
(Behavioural) Finance:
Data vendors:
Acceptance of new technologies through commercial activities
This project is licensed under the MIT License.