A Cross-Domain Data Hub to Track the Impact of COVID-19 on U.S. Electricity Markets

This data hub, COVID-EMDA+ (Coronavirus Disease - Electricity Market Data Aggregation+), is specifically designed to track the potential impacts of COVID-19 on the existing U.S. electricity markets. Many different data sources are merged and harmonized here in order to enable further interdisciplinary researches.

Based on the data hub, we have developed a supporting toolbox to realize a series of advanced analysis. Both Python and Matlab versions are available, and read more at this link.

Features

• Overall, this data hub contains the coronavirus case & deaths, weather, generation mix, load and price data for all the existing U.S. electricity marketplaces (CAISO, MISO, ISO-NE, NYISO, PJM, SPP , ERCOT) and some typical cities in these markets (Los Angeles, Boston, New York City, Philadelphia, Chicago, Kansas City, Houston). We also integrate other additional resources, including the mobile device location, night-time lighting images, load forecasting, congestion price, forced outage and renewable curtailment data. Historical data dating back to 2017 are included as time-series benchmarks.
• This data hub is updated every day after careful quality control. All data are carefully verified and coordinated to match the geological scale. All data are recorded and tidied in a consistent, compact and ready-to-use data format that makes it easy for cross-market analysis.
• Some useful parsers as well as supplementary resources are provided for other user-defined extensions.
• A supporting toolbox written in Python and Matlab is realized to make the data-driven analysis as easy as possible.

Navigation

This data hub mainly contains five components: source data, released data, supplementary resources, parser codes, and quick-start tutorials. We navigate this data hub as follows.

All the data source files are archived in folder date_source/, the cleaned and processed data are stored in folder data_release/. The support team will conduct daily updates to capture the latest data. All these files are properly collected by location. The file naming convetion for this data hub is: MARKET_AREA_CATEGORY.csv, e.g. nyiso_nyc_load.csv is a dataset of load profile in New York City from 2017 to present.

Readers can turn to folder startup/ for quick start, supplementary/ for addtional data and codes in our research work, and parser/ for the basic tools to handle the source files.

Suggested Citation

• Please cite the following paper when you use this data hub:
  G. Ruan, D. Wu, X. Zheng, H. Zhong, C. Kang, M. A. Dahleh, S. Sivaranjani, and L. Xie, ``A Cross-Domain Approach to Analyzing the Short-Run Impact of COVID-19 on the U.S. Electricity Sector,'' Joule, vol. 4, pp. 1-16, 2020.
  This paper conducts a comprehensive introduction of this data hub and further analysis results for electricity demand across the U.S.
  Available at: arXiv and EnerarXiv.
• Other research studies of our group are recommended:
  G. Ruan, J. Wu, H. Zhong, Q. Xia, and L. Xie, ``Quantitative Assessment of U.S. Bulk Power Systems and Market Operations during the COVID-19 Pandemic,'' Applied Energy, vol. 286, pp. 116354, 2021.
  This paper substantiates the pandemic's impacts from the perspectives of power system security, electric power generation, electric power demand and electricity prices.
  Available at: EnerarXiv.
  G. Ruan, Z. Yu, S. Pu, S. Zhou, H. Zhong, L. Xie, Q. Xia, and C. Kang, ``Open-Access Data and Toolbox for Tracking COVID-19 Impact on Power Systems,'' IEEE Trans on Power Systems, 2022 (Accepted).
  This paper gives a comprehensive introduction of the supporting toolbox (both Python and Matlab version), most of the methodologies, implementation details, and three real-world empirical cases are discussed.
  Available at: arXiv and TechRxiv.
  H. Zhong, Z. Tan, Y. He, L. Xie, and C. Kang, ``Implications of COVID-19 for the Electricity Industry: A Comprehensive Review,'' CSEE Journal of Power and Energy Systems, 2020.
  This paper provides a review of the global impacts that COVID-19 has caused on the electricity sector.
  Available at: JPES.

Data Source

This data hub contains five major components: U.S. electricity market data, public health data, weather data, mobile device location data, and satellite images. For some categories, multiple data sources are carefully gathered to ensure accuracy.

• Electricity Market Data:
  Description:   This part includes the generation mix, metered load profiles and day-ahead locational marginal prices data. We also include the day-ahead load forecasting, congestion price, forced outage and renewable curtailment data as the supplementary source.
  Link:   CAISO,   MISO,   ISO-NE,   NYISO,   PJM,   SPP,   ERCOT,   EIA,   EnergyOnline.

• Public Health Data:
  Description:   This part includes the COVID-19 confirmed cases, deaths data, infection rate and fatal rate. We aggregate and fine-tune the data to market and city levels.
  Link:   John Hopkins CSSE.

• Weather Data:
  Description:   This part includes temperature, relative humidity, wind speed and dew point data. Typical weather stations are selected according to their geological locations and data quality.
  Link:   Iowa State Univ IEM,   NOAA.

• Mobile Device Location Data:
  Description:   This part includes social distancing data and patterns of visits to Point of Interests (POIs). These data are derived by aggregating and processing the real-time GPS location of cellphone users by Census Block Group. To obtain the access to the original data, please click the link below and apply for SafeGraph's permission (totally free).
  Link:   Mobility Data from SafeGraph

• Night Time Light (NTL) Satellite Data:
  Description:   This part includes the raw satellite image taken at night time in each area.
  Link:   NTL Images from NASA

Support Team

This project is a collaboration of our group members under the supervision of Prof. Le Xie at Texas A&M University. The support team keeps processing, correcting and updating the data every day. The team will also conduct further research analysis and share the latest progress in this repository.

Please also check our Group Website for the detailed biography of each group member.

Contact Us

Please contact us if you need further technical support or search for cooperation. Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Email contact:   Le Xie,   Dongqi Wu,   Xiangtian Zheng,   Jiahan Wu.