NewPipeExtractor Versions Save

⚠️ This version contains breaking changes (thus the bump to 0.24):

Due to the removal of the usage of InnerTube API keys, some methods in YoutubeParsingHelper which can be used by clients have been changed in #1168:

• areHardcodedClientVersionAndKeyValid was renamed to isHardcodedClientVersionValid
• getKey was removed, since there is no InnerTube API key anymore
• resetClientVersionAndKey was renamed to resetClientVersion
• isHardcodedYoutubeMusicKeyValid was renamed to isHardcodedYoutubeMusicClientVersionValid
• getYoutubeMusicKey was renamed to getYoutubeMusicClientVersion and now just returns the version instead of (key, name, version), since there is no key anymore and the name is fixed ("67") and therefore hardcoded
• other related methods which are not really meant to be used outside of the extractor

New

• [YouTube] Implement emergency meta info #1135
• [YouTube] Add Releases channel tab as an extractor albums channel tab #1147
• [PeerTube] Add support for v6 features (frames and segments) #1142

Improved

• [YouTube] Detect new account termination messages for YouTube #1160
• Make getCommentText return @Nonnull values #1162
• [YouTube] Use accessibility data for social links in attributed descriptions when present #1163

Fixed

• [PeerTube] Fix local kiosk #1144
• [PeerTube] Do not accept non-URLs #1143
• [MediaCCC] Allow obtaining channel tab link handler #1148
• [YouTube] Support new comments data #1163
• [YouTube] Update clients versions, restore access to some streams and more #1168

Development

• Bump org.jsoup:jsoup from 1.16.2 to 1.17.2 #1145
• Bump com.github.spotbugs:spotbugs-annotations from 4.8.0 to 4.8.3 #1138
• Bump actions/setup-java from 3 to 4 #1132
• Bump org.junit:junit-bom from 5.10.0 to 5.10.2 #1156
• Bump actions/cache from 3 to 4 #1153
• Bump actions/upload-artifact from 3 to 4 #1139
• LocaleCompat.forLanguageTag: return Optional if parsing fails #1151

Fixed

• Fix SearchInfo's non-null MetaInfo being null when initialized or when an extraction error occurs #1141
• [YouTube] Always return -1 as duration of Shorts returned inside reel items, since YouTube does not provide duration for Shorts anymore #1140

⚠️ This version contains breaking changes (thus the bump to 0.23):

• Channel tabs support (more details in #1082):
  • ChannelExtractor is now only an Extractor (and not a ListExtractor)
  • All link handlers for the channel tab extractors of a specific channel can be obtained through ChannelExtractor.getTabs()
  • ChannelTabExtractor does inherit ListExtractor<InfoItem>, and thus can return items of different types, and not only videos
• Multiple images support (more details in #889):
  • All extractor methods that used to return image urls have been renamed from get*Url to get*s (e.g. getThumbnailUrl -> getThumbnails)
  • Those methods return a List<Image>, and every image contains not only the url, but also data about the resolution

New

• Add support for channel tabs and channel tags #1082 #1092
• [YouTube] Support more channel headers #1094
• [YouTube] Support Shorts UI in playlists #1093
• Multiple images support #889 #1113
• [YouTube] Add support for ultralow audio formats #1063
• Add hasCreatorReply() to CommentsInfoItem #1111
• [YouTube] Add channel owner to CommentsInfoItem: isChannelOwner() #1114

Improved

• [YouTube] Improve and fix YoutubeJavaScriptExtractor #1087 #1091
• [YouTube] Refactor JavaScript usage and fix extraction of obfuscated signature deobfuscation function #1108
• [YouTube] Make some improvements and fixes #1127

Fixed

• [media.ccc.de] Only extract kiosk live stream rooms if they are streaming #1089
• [SoundCloud] Detect whether there are any more search results #1081

Development

• Replace cryptographically secure random with regular random #1088
• [Bandcamp] Fix StreamExtractorTest #1105
• Use Locale.forLanguageTag() in tests #1107
• Update copyright notices to use correct GPLv3 format #1110
• Dependency updates:
  • Bump org.junit:junit-bom from 5.9.3 to 5.10.0 #1083
  • Bump actions/checkout from 3 to 4 #1103
  • Bump org.jsoup:jsoup from 1.16.1 to 1.16.2 #1117

New

• [YouTube] Add support for CarouselHeaderRenderer #1050 #1086
• Extract playlists description #1061

Improved

• [PeerTube] Don't return "No description" when there is no description for a channel or an account #1055
• Add more audio media formats and MediaFormat.getAllForMimeType(mimeType) #1074
• [YouTube] Check whether player responses are valid for all InnerTube clients used #1070
• [YouTube] Add support for AV1 itags #706

Fixed

• [Bandcamp] Handle paywalled tracks #1033
• [YouTube] Fix YouTube Music client version extraction from HTML page #1051
• [PeerTube] Fix multi level comment replies #1052
• [YouTube] Extract trends from A/B tested "Videos" tab and fix extraction of trends name from A/B tested new title design #1045
• [YouTube] Fix parsing short relative date formats (English only) #1068
• [YouTube] Support multiple declarations for throttling parameter function name array #1076
• [YouTube] Workaround again 403 HTTP issues on the ANDROID InnerTube client and fix stream tests #1084

Development

• Bump junit-bom from 5.9.2 to 5.9.3 #1053
• Make sure CI is not vulnerable to bash word splitting #1054
• Bump jsoup from 1.15.4 to 1.16.1 #1058
• Fix BandcampRadioStreamExtractorTest.testGetAudioStreams() #1072
• Initialize services at the correct place #1071
• Add ProGuard rules to README #1080

New

• Add track types and locale properties to audio streams #1026 #1041

Improved

• [Bandcamp] Support loading additional comments #1030

Fixed

• [Bandcamp] Show comments as disabled on radio streams #1031
• [YouTube] Fix partial non-extraction of "No views" string in stream items #1029
• [YouTube] Fix hashtags links extraction and escape HTML links #1032
• [YouTube] Fix ParsingException when comments are unavailable in a video #1040
• [Bandcamp] Fix extraction of related playlist items URL #1047

Development

• Update JDK to 11 in JDoc workflow #1038
• [PeerTube] Fix comments info test #1034

:warning: Important note: Starting with this version, NewPipe Extractor requires Java 11 to be built.

New

• Add support for extracting audio track information #973
• [YouTube] Extract timestamps and URLs in comments #928
• [PeerTube] Support comment replies #941
• [PeerTube] Support searching for playlists and channels #996
• [YouTube] Add support to extract total comment count #794 #1014

Improved

• Use Description object for comments text. #987 #991
• [YouTube] Implement bold/italic/strike-through support #990 #1005 #1006
• [YouTube] Improve YoutubeStreamInfoItemExtractor #1000
  • make duration of video premieres available
  • returns view count and upload date of streams in playlists
• Add compat Locale.forLanguageTag() implementation #910
• [PeerTube] Report Exceptions thrown while getting a stream's subtitles #994
• [SoundCloud] Implement getUploaderUrl() and isUploaderVerified() for PlaylistInfoItemExtractor #995
• [YouTube] Support live URLs #1022

Fixed

• [YouTube] Fix extracting the detailed error message for unavailable streams #980
• [YouTube] Fix search subscriber count extraction with channel handles #978
• [YouTube] Remove topStandaloneBadge check for view count of stream items #1016
• [YouTube] Fix NPE in search when getting channel items without subscriber count #1020
• [YouTube] Now music mixes can be treated as normal mixes #1013
• [YouTube] Fix channel ID extraction of YouTube channels RSS feeds #1015
• [SoundCloud] Fix extraction of tracks like count #1024
• [YouTube] Fix getting the comment text if the comment contains a hashtag #1021

Development

• Use immutable sets in YoutubeParsingHelper and bump the required Java version to 11 #893
• Bump gson from 2.9.1 to 2.10.1 #954 #1009
• Use StandardCharsets.UTF_8. #880
• Use JDK 11 for Jitpack gradle builds #975
• [YouTube] Simplify duration parsing #971
• Update Checkstyle from 9.3 to 10.4 #985
• Remove unused autolink dependency #984
• Block wrong nullable/nonnull imports in Checkstyle #989
• Improve Utils.isBlank(String) method #992
• Use immutable Map factory methods #986
• Bump junit-bom from 5.9.1 to 5.9.2 #1010
• Bump jsoup from 1.15.3 to 1.15.4 #1028

NOTE: This pre-release has been created to keep track of what Extractor changes went in which NewPipe release. That's its only purpose. The source zips under Assets DO NOT match the changes listed below.

New

• [YouTube] Add comment reply count support #936
• [YouTube] Add uploaderUrl() and uploaderVerified() to PlaylistInfoItem #959
• [YouTube] Support handles #964

Improved

• [YouTube] Support new metadata format of playlists #958
• [YouTube] Add support for attributed text description #953

Fixed

• [media.ccc.de] Avoid possible NullPointerException in MediaCCCRecentKiosk #930
• [SoundCloud] Fix SoundcloudPlaylistExtractor: tracks are in correct order #939
• [SoundCloud] Fix getting more comments #943
• [YouTube] Workaround 403 HTTP errors of ANDROID client streams #960
• [YouTube] Support richGridRenderer on channel page to fix extraction of videos in channels #944
• [YouTube] Support all channel usernames #964

Development

• Bump spotbugs-annotations from 4.7.1 to 4.7.3 #924 #945
• Bump junit-bom from 5.9.0 to 5.9.1 #931
• Use EnumMap in PatternsHolder #888

NOTE: This pre-release has been created to keep track of what Extractor changes went in which NewPipe release. That's its only purpose. The source zips under Assets DO NOT match the changes listed below.

Improved

• [YouTube] Get original untranslated title #857
• [YouTube] Return mqdefault thumbnails in fast feed #927
• [YouTube] Add JavaScript lexer to parse completely throttling decryption function #905

Fixed

• [YouTube] Fix throttling parameter decryption #891 #912 #925, thanks to @Theta-Dev!
• [YouTube] Update clients version and improve extraction of API key and client version of WEB client, fixing video and audio streams replacement by the ones of a video saying The following content is not available on this app #890
• Fixed all tests and the corresponding code #882
• [YouTube] Fixes mixes playlist extraction in some countries (EU) by allowing YoutubeParsingHelper#consentAccepted to be set to true #882
• [YouTube] Fix extraction of like count #926
• [YouTube] Fix extraction of the Trending kiosk #920
• [YouTube] Fix fallback throttling decryption function regex #934

Development

• Bump spotbugs-annotations from 4.7.0 to 4.7.1 #867
• Bump jsoup from 1.15.1 to 1.15.3 #870 #911
• Use minimum required permissions for GitHub workflows #869
• Bump junit-bom from 5.8.2 to 5.9.0 #879
• Use Objects.requireNonNull() #877
• Use Collections methods #878
• Bump gson from 2.9.0 to 2.9.1 #883
• Use String.join() and Collectors.joining() #881
• Make improvements to methods using toArray() #886
• [YouTube] Remove deprecated throttling decryption code #901
• [YouTube] Checks if the throttling parameter decryption function is valid JavaScript code #882
• Update Gradle wrapper to 7.5.1 #903
• Remove EMPTY_STRING #907

NOTE: This pre-release has been created to keep track of what Extractor changes went in which NewPipe release. That's its only purpose. The source zips under Assets DO NOT match the changes listed below.

New

• Support delivery methods other than progressive HTTP #810 #859 #864

Improved

• [Bandcamp] Extract stream length #839

Fixed

• [YouTube] Fix extracting YouTube Music client version and key in the EU #848

Development

• Cleanup TimeAgoParser #821
• Cleanup of YoutubeStreamExtractor and some related classes #782
• Fix some warnings and in Bandcamp use Java 8 streams #822
• Remove Checkstyle suppressions file and fix too-long links #843
• Correct PeertubePlaylistExtractorTest unit tests #860
• Remove unused methods #846
• Bump jsoup from 1.14.3 to 1.15.1 #847
• Bump spotbugs-annotations from 4.6.0 to 4.7.0 #845
• Update User-Agent just for tests #838

Fixed

• [YouTube] Use correct order of clients to get Android streams first #841

Development

• Bump actions/setup-java from 2 to 3 #828
• Update User-Agent (for tests) #838