Generalized data analysis workflow via a consistent easy to use interface.
The Python Satellite Data Analysis Toolkit (pysat) v3.2.0 adds multiple new features for data support, including custom data set concatenation and improved metadata support.
concat_data
methods.include
kwarg to Instrument.concat_data
to expand allowed inputs.__getitem__
method and reduced
memory usage in the load
method._get_epoch_name_from_data
to
reduce code duplication.__delitem__
to Meta and drop
to MetaHeader and MetaLabels classes.Meta.drop
to traverse attached MetaLabel and MetaHeader data.__delitem__
and drop
to Instrument and Constellation classes.pysat_ndtesting
instrument with sample rate,
shift in time.pysat_ndtesting
.utils.registry.register_by_module
.utils.files
.utils.files.parse_fixed_width_filenames
and utils.files.parse_delimited_filenames
functionsmeta['data_var', 'label'] = value
structure.eval_warnings
to support testing against multiple warning types.utils.files.parse_fixed_width_filenames
output for empty file listutils.files
to consider type specifiers
when identifying appropriate files in a directoryConstellation.to_inst
method definition of coords, using dims
to combine common dimensions instead.pysat.utils.files
in other modules.pysat_testing2d
instrumentpysat_testing_xarray
instrumentpysat_testing2d_xarray
instrumentinstrument_test_class
malformed_index
kwarg in test instrumenntsconvert_timestamp_to_datetime
function_test_download_travis
flagfreq
kwarg from download
use_header
kwarg from load
and changed default
behaviour to use_header=True
labels
kwarg for pysat.Instrument()
utils.load_netcdf4
method_filter_netcdf4_metadata
methodto_netCDF4
_new_tests
flag for packages to ignore select new testsThe Python Satellite Data Analysis Toolkit (pysat) v3.1.0 is a minor release with significant improvements in the base classes and data I/O. This includes improvements in Constellations, data stored in the xarray Dataset format, and saving Instrument data in netCDF files.
to_inst
method to the Constellation classexport_pysat_info
kwarg to to_netcdf
routines to select whether
pysat instrument info is written to filesprint
on an instrumentcheck_and_make_path
to treat an empty path as the current dirmeta_kwargs
attribute and kwarg to Instrument, allowing full custom
specification of the Meta class on instantiationdata_types
input to Meta and certain MetaLabels methods, allowing
the default values to be set to the specified data type when multiple types
are allowed, ensure these are updated when adding new data to an Instrument_update_label_types
to MetaLabels, expanding the Python float/int
types to include all numpy float/int typesstrict_dim_check
for loading xarray Datasets through netCDFcombine_by_coords
kwarg to io.load_netcdf
for use on multi-file
xarray Datasetslabels
in favor of meta_kwargs
and
replaced the meta_labels
attribute with the meta_kwargs
attributelabels
keyword arg in favor of meta_kwargs
in the
netCDF I/O functions and Instrument sub-modulemalformed_index
kwarg in the test instruments. This is
replaced by non_monotonic_index
and non_unique_index
pysat.instruments.methods.general.list_files
to handle file
cadences other than daily or monthlypysat.Instrument.bounds
to handle all input types for step
and width
regardless of start
and stop
time. Also fixed
seasonal bounds specified using file names.pysat.utils.io.apply_table_translation_to_file
check for duplicates
in the meta translation tablepysat.utils.files
malformed_index
kwarg in the test instruments. This is
replaced by non_monotonic_index
and non_unique_index
instruments.pysat_testing
tag='no_download' to return an empty
pandas DataFrame for load
constellations.testing_partial
, which loads a partially empty
constellation datasetThe Python Satellite Data Analysis Toolkit (pysat) v3.0.6 is a bug fix release to improve compatibility with Python 3.6 and appropriate versions of pandas and xarray.
pysat_testing2d_xarray
instrument deprecated and replaced by
pysat_ndtesting
The Python Satellite Data Analysis Toolkit (pysat) v3.0.5 is a bug fix release, including improved compatibility with pandas v1.5.
The Python Satellite Data Analysis Toolkit (pysat) v3.0.4 is a bug fix release that improves compatibility with xarray when accessing data via slices.
methods.testing.generate_times
.The Python Satellite Data Analysis Toolkit (pysat) v3.0.2 is primarily intended as a bugfix and maintenance release. However, this release does include expanded support for writing and reading netCDF4 files for user customizable file standards.
empty_partial
to the Constellation classinst_module
pysat.utils.testing
functionsstart_time
keyword for test instrumentsmax_latitude
keyword for non-model test instruments_test_download_ci
as a standard attribute for pysat.Instrument
pysat.instruments.pysat_testmodel
as tag='pressure_levels'._test_load_opt
instrument attributerename
and add_epoch_metadata
methods to the Meta classpysat.utils.testing
.pysat.utils.io.load_netcdf
.Meta.var_case_name
and Meta.attr_case_name
to support
list of str as input as well as str.Instrument.rename
to take a fuction or mapping dict as input,
after adapting routine to use Meta.rename
pysat.utils.files
.pysat.Instrument.pandas_format
so that changes to setting
are immediately reflected by changes to Instrument object.pysat.utils.io.load_netcdf
.pysat.Meta
instance.meta.to_dict
to support creating more compliant
netCDF files.meta.default_to_netcdf_translation_table
,
default_from_netcdf_translation_table
and
apply_table_translation
to pysat.utils.io
to support
improved compatability of pysat netcdf files without user code
changes.drop_meta_labels
kwarg to pysat.utils.io.load_netcdf
to
support easy removal of unwanted metadata during loading.meta_processor
in pysat.utils.io.load_netcdf
and pysat.utils.io.inst_to_netcdf
enabling developers to easily modify
metadata before it is loaded from/written to storage.pysat.utils.io.inst_to_netcdf
.pysat.utils.io.meta_array_expander
.pysat.Instrument.vars_no_time
function to return data variables
excluding the main time index.decode_times
to flag using xarray or pysat processing
of times when loading a file via pysat.utils.io.load_netcdf
.pysat.utils.io.xarray_all_vars
which returns a list of all variables,
coordinates, and dimensions.pysat.utils.scale_units
assert_isinstance
and assert_hasattr
to pysat.utils.testing
freq
as a standard kwarg for pysat.Instruments.download
_test_download_travis
as a standard attribute for
pysat.Instrument
. The function is replaced by _test_download_ci
pysat.tests.instrument_test_class.initialize_test_inst_and_date
,
which is moved to pysat.tests.classes.cls_instrument_library
.pysat.tests.instrument_test_class.InstTestClass
, which is
replaced by pysat.tests.classes.cls_instrument_library.InstLibTests
.pysat.Instrument._filter_netcdf4_metadata
and replaced it
with pysat.utils.io.filter_netcdf4_metadata
.fname
from a kwarg to an arg in pysat.Instruments.to_netcdf4
pysat.instruments.pysat_testing_xarray
pysat.Instrument.generic_meta_translator
which is replaced
by pysat.utils.io.apply_table_translation_to_file
.convert_timestamp_to_datetime
,
which is replaced by new functionality in load_netcdf
.listify
docstringREADME.md
to display in more environments.tag
and inst_id
keywords and docstrings to new standardspysat.utils.load_netcdf4
parse_delimited_filename
output consistency and ability to handle
leading and trailing non-parsed text in filenames (e.g., file extensions)decode_timedelta=False
for loading xarray from netcdf4 (#823)pysat.Instrument._iter_list
tag
and inst_id
to be specified as None (#892)pysat.utils.time.create_datetime_index
(#906)pysat.utils.scale_units
and ensured '/cc' compatibility.inst_module
and platform
/name
are used to
instantiate an instrument (#850). In case of this, inst_module
takes
priority.pysat.Instrument()
(#851)Instrument.data
type in
utils.coords.update_longitude
from_os
now always returns a sorted Series.pysat.Instrument.generic_meta_translator
to use
pysat.Meta.to_dict
and the latest metadata label standards.pysat.params['data_dirs']
.pysat.Instrument._get_data_info
.pysat.utils.time
pysat.utils.file.parse_delimited_filename
test_orbits
methods.testing
pysat.Instrument
class into multiple filespysat.Instrument
with pytest.mark.parametrizeInstLibTests
more portable to streamline user implementation of the
standard end-to-end instrument tests.pysat.tests.classes
test_meta
to search all warnings, not just the firstdata_path
check from pysat.instruments.methods.general.list_files
The Python Satellite Data Analysis Toolkit (pysat) v3.0.1 is primarily intended as a bugfix and maintenance release. Most significantly, pysat passes online unit tests on the Windows platform.
tag
and inst_id
at pysat.Instrument
instantiation. (#784)empty
, index
, date
, today
, yesterday
,
tomorrow
, and variables
(#764)The Python Satellite Data Analysis Toolkit (pysat) v3.0.0 release is a substantive release intended to support pysat in operational environments as well as expand community oriented capabilities for space science data analysis. Core features have been retained within pysat while supporting instruments and analysis codes have been moved to individual packages. While v3.0.0 is not backwards compatible due to a reorganization of some functions and the data storage structure, the user interface is largely unchanged.
_test_download
, _test_download_travis
, _password_req
strict_time_flag
now defaults to Truewarn_empty_file_list
to pysat.params to provide a warning if no
Instrument files are found. Default is False.directory_format
input to Instrument
to be a functionstop
is an inclusive bound, end
is
exclusivesat_id
Instrument keyword argument to inst_id
inst_id
for number of simulated samples for test
instrumentspysat.instruments.methods.general.list_files
kwarg
fake_monthly_files_from_daily
to file_cadence
default
to preprocess
pysat.data_dir
. Information now at pysat.params['data_dirs']
.pysat.Instrument._filter_datetime_input
to
pysat.utils.time.filter_datetime_input
pysat.utils.set_data_dir
name
kwarg in Constellation to const_module
labels
tag
and inst_id
behaviour in testing instruments__str__
and __repr__
functions for basic classeseval(inst.__repr__())
(#636)calc_solar_local_time
for data sets with longitude coordinatesinst_module
(#728)eval(Instrument.__repr__())
works in more casespysat.utils.files.check_and_make_path
dtype
for empty pandas.Series for forward compatibilityacknowledgements
and references
The Python Satellite Data Analysis Toolkit (pysat) v2.3.0 release is intended to be the last release of the v2.x series and the last release to include support for Python 2.x. pysat v2.3.0 primarily features DeprecationWarnings for pysat functions that are changing as part of the 3.0.0 release.
inst_id
(replaces sat_id
)file_cadance
pysat.params['data_dirs']
, the replacement
for pysat.data_dir
in pysat 3.0.0sat_id
, default
, multi_file_day
,
manual_org
, and _filter_datetime_input
, data_dir
)fake_montly_files_from_daily
sat_id
name
_files
classssnl
modulesat_id
to specify number of points
in test instruments