Python module for Dataverse Software (dataverse.org).
timeout
to None
to avoid timeout errors by @JR-1991 in https://github.com/gdcc/pyDataverse/pull/188
LICENSE.txt
by @JR-1991 in https://github.com/gdcc/pyDataverse/pull/185
Full Changelog: https://github.com/gdcc/pyDataverse/compare/v0.3.2...v0.3.3
We are excited to announce the release of the latest patch version of pyDataverse after a significant period of inactivity. This update brings a range of new functionalities and bug fixes, aimed at improving the stability and performance of the pyDataverse library.
The library has been equipped with a CI/CD pipeline to ensure consistent integration with Dataverse. To achieve this, we have utilized the Dataverse Action which uses the progress made by the Dataverse Containerization Working Group to create local instances of Dataverse with ease. This has made contributions safer and made it easier to test pull requests.
PyDataverse has also switched from the requests
library to HTTPX, a powerful library for performing HTTP requests. The library offers better performance and compatibility and allows async requests, which were previously impossible. For more information on how to use the new async, please refer to PR https://github.com/gdcc/pyDataverse/pull/175 for now.
Finally, pyDataverse's building and dependency management has been transferred to pyproject.toml
from setup.py
. The current de facto standard in packaging Python projects offers numerous advantages over setup.py
while maintaining compatibility with the pip
installer.
active
by @JR-1991 in https://github.com/gdcc/pyDataverse/pull/168
httpx
by @JR-1991 in https://github.com/gdcc/pyDataverse/pull/174
pyproject.toml
and poetry
by @JR-1991 in https://github.com/gdcc/pyDataverse/pull/180
Full Changelog: https://github.com/gdcc/pyDataverse/compare/0.3.1...v0.3.2
If you are interested in the development of pyDataverse, we invite you to join us for a chat on our Zulip Channel. This is the perfect place to discuss and exchange ideas about the development of pyDataverse. Whether you need help or have ideas to share, feel free to join us!
We have formed a pyDataverse working group to exchange ideas and collaborate on pyDataverse. There is a bi-weekly meeting planned for this purpose, and you are welcome to join us by clicking the following WebEx meeting link. For a list of all the scheduled dates, please refer to the Dataverse Community calendar.
Small bugfix of #126.
For help or general questions please have a look in our Docs or email [email protected].
Thanks to Karin Faktor for finding the bug.
PyDataverse is supported by AUSSDA and by funding as part of the Horizon2020 project SSHOC.
This release is a big change in many parts of the package. It adds new API's, re-factored models and lots of new documentation.
Overview of the most important changes:
get_children()
jsonschemas
required)update_datafile()
Version 0.3.0 is named in honor of Ruth Wodak (Wikipedia), an Austrian linguist. Her work is mainly located in discourse studies, more specific in critical discourse analysis, which looks at discourse as a form of social practice. She was awarded with the Wittgenstein-Preis, the highest Austrian science award.
For help or general questions please have a look in our Docs or email [email protected].
The new functionalities were developed with some specific use-cases in mind:
See more detailed in our Documentation.
Retrieve data structure and metadata from Dataverse instance (DevOps)
Collect all Dataverses, Datasets and Datafiles of a Dataverse instance, or just a part of it. The results then can be stored in JSON files, which can be used for testing purposes, like checking the completeness of data after a Dataverse upgrade or migration.
Upload and removal of test data (DevOps)
For testing, you often have to upload a collection of data and metadata, which should be removed after the test is finished. For this, we offer easy to use functionalities.
Import data from CSV templates (Data Scientist)
Importing lots of data from data sources outside dataverse can be done with the CSV templates as a bridge. Fill the CSV templates with your data, by machine or by human, and import them into pyDataverse for an easy mass upload via the Dataverse API.
long_description_content_type
(#4)Summary: Add other API's next to Native API and update Native API.
get_datafile()
, get_datafiles()
, get_datafile_bundle()
)request_access()
, allow_access_request()
, grant_file_access()
, list_file_access_requests()
)total()
, past_days()
, get_dataverses_by_subject()
, get_dataverses_by_category()
, get_datasets_by_subject()
, get_datasets_by_data_location()
get_service_document()
search()
get_children()
)dataverse_id2alias()
)get_dataverse_contents()
)get_dataverse_assignments()
)get_dataverse_facets()
)edit_dataset_metadata()
) (#19)destroy_dataset()
)create_dataset_private_url()
, get_dataset_private_url()
, delete_dataset_private_url()
)get_dataset_versions()
, get_dataset_version()
)get_dataset_assignments()
)get_dataset_lock()
)get_datafiles_metadata()
update_datafile_metadata()
)redetect_file_type()
)restrict_datafile()
)reingest_datafile()
, uningest_datafile()
)upload_datafile()
)replace_datafile()
get_dataverse_roles()
, create_role()
, show_role()
, delete_role()
)get_user_api_token_expiration_date()
, recreate_user_api_token()
, delete_user_api_token()
)get_user()
) (#59)get_info_api_terms_of_use()
)create_dataset()
(#3)upload_datafile()
)pydataverse
auth
parameter used anymore.Summary: Re-factoring of all models (Dataverse, Dataset, Datafile).
New methods:
from_json()
imports JSON (like Dataverse's own JSON format) to pyDataverse models objectget()
returns a dict of the pyDataverse models objectjson()
returns a JSON string (like Dataverse's own JSON format) of the pyDataverse models object. Mostly used for API uploads.validate_data()
validates a pyDataverse object with a JSON schemawrite_dicts_as_csv()
) (#11)get_children()
and extract Dataverses, Datasets and Datafiles (dataverse_tree_walker()
)dataverse_tree_walker()
in seperate JSON files (save_tree_data()
)validate_data()
)clean_string()
)create_dataverse_url()
, create_dataset_url()
, create_datafile_url()
)read_csv_to_dict()
: replace dv.
prefix, load JSON cells and convert boolean cell stringsMany new pages and tutorials:
Thanks to Daniel Melichar (@dmelichar), Vyacheslav Tykhonov (Slava), GDCC, @ecowan, @BPeuch, @j-n-c and @ambhudia for their support for this release. Special thanks to the Pandas project for their great blueprint for the Contributor Guide.
PyDataverse is supported by funding as part of the Horizon2020 project SSHOC.
This release fixes a bug in the Dataset.dict()
generation.
For help or general questions please have a look in our Docs or email [email protected].
series
, socialScienceNotes
and targetSampleSize
caused error in Dataset.dict()
, cause the contained sub-values were stored directly in own class-attributes.To find out how you can contribute, please have a look at the Contributor Guide. No contribution is too small!
The most important contribution you can make right now is to use the module. It would be great, if you install it, run some code on your PC and access your own Dataverse instance if possible - and give feedback after it (contact).
pyDataverse includes a collection of functionalities to import, export and manipulate data and it's metadata via the Dataverse API.
-- Greetz, Stefan Kasberger
This release adds functionalities to import, manipulate and export the metadata of Dataverses, Datasets and Datafiles.
Version 0.2.0 is named in honor of Ida Pfeiffer (Wikipedia), an Austrian traveler and travel book author. She went on for several travels around the world, where she collected plants, insects, mollusks, marine life and mineral specimens and brought most of them back home to the Natural History Museum of Vienna.
For help or general questions please have a look in our Docs or email [email protected].
dict()
Api()
(PR #8)dict()
for automatic import of datasets into a Dataset()
objecttox.ini
requests>=2.12.0
or newer neededFrom 18th to 22nd of June 2019, pyDataverse's main developer Stefan Kasberger will be at the Dataverse Community Conference in Cambridge, MA to exchange with others about pyDataverse end develop it further. If you are interested and around, drop by and join us. If you can not attend, you can connect with us via Dataverse Chat.
To find out how you can contribute, please have a look at the Contributor Guide. No contribution is too small!
The most important contribution you can make right now is to use the module. It would be great, if you install it, run some code on your PC and access your own Dataverse instance if possible - and give feedback after it (contact).
Another way is, to share this release with others, who could be interested (e. g. retweet my Tweet, or send an Email).
pyDataverse includes a collection of functionalities to import, export and manipulate data and it's metadata via the Dataverse API.
https://twitter.com/stefankasberger/status/1140832352517668864
Thanks to Ajax23 for the PR #8. Great contribution, and it's always amazing to see the idea of Open Source in action. :)
-- Greetz, Stefan Kasberger
This release is a quick bugfix. It adds requests to the install_requirements and updates the packaging and testing configuration.
For help or general questions please have a look in our Docs or email [email protected].
requests
to the install_requires
in setup.py
setup.py
tools/tests-requirements.txt
tox.ini
: add python versions, add dist test, add pypitest test, clean up and re-structure configurationTo find out how you can contribute, please have a look at the Contributor Guide. No contribution is too small!
The most important contribution right now is simply to use the module. It would be great, if you install it, run some code on your PC and access your own Dataverse instance if possible - and give feedback after it (contact).
pyDataverse includes the most basic data operations to import and export data via the Dataverse API. The functionality will be expanded in the next weeks with more requests and a class-based data model for the metadata. This will allow to easily import and export metadata, and upload it directly to the API.
Thanks to @moumenuisawe for mentioning this bug.
-- Greetz, Stefan Kasberger
This release is the initial, first one of pyDataverse. It offers basic features to access the Dataverse API via Python, to create, retrieve, publish and delete Dataverses, Datasets and Datafiles.
Version 0.1.0 is named in honor of Marietta Blau (Wikipedia), an Austrian researcher in the field of particle physics. In 1950, she was nominated for the Nobel prize for her contributions.
For help or general questions please have a look in our Docs or email [email protected].
api.py
:
utils.py
: File IO and data conversion functionalities to support API operationsexceptions.py
: Custom exceptionstests/*.py
: Tests with test data in pytest, tested with tox on travis ci.To find out how you can contribute, please have a look at the Contributor Guide. No contribution is too small!
The most important contribution right now is simply to use the module. It would be great, if you install it, run some code on your PC and access your own Dataverse instance if possible - and give feedback after it (contact).
Another way is, to share this release with others, who could be interested (e. g. retweet my Tweet, or send an Email).
pyDataverse includes the most basic data operations to import and export data via the Dataverse API. The functionality will be expanded in the next weeks with more requests and a class-based data model for the metadata. This will allow to easily import and export metadata, and upload it directly to the API.
Thanks to dataverse-client-python, for being the main orientation and input for the start of pyDataverse. Also thanks to @kaczmirek, @pdurbin, @djbrooke and @4tikhonov for their support on this.
-- Greetz, Stefan Kasberger