Major Changes

• Reworked lists implementation that decreased memory usage of orders of magnitude
• Improved code robustness via extensive code fuzzing
• Various improvements to overall library performance
• Extended IPv6 support

New Supported Protocols and Services

• Add "Heroes of the Storm" video game signature detection. (#1949)
• Add Apache Thrift protocol dissector. (#2007)
• Add Remote Management Control Protocol (RMCP).
• Add Service Location Protocol dissector. (#2036)
• Add VK detection (#1880)
• Add Yandex services detection (#1882)
• Add a new protocol id for generic Adult Content traffic (#1906)
• Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904)
• Add bitcoing protocol dissector. (#1992)
• Add detection of Roblox games (#2054)
• Add support for (un-encrypted) HTTP/2 (#2087)
• Add support for Epic Games and GeForceNow/Nvidia (#1990)
• Add support for SRTP (#1977)
• Added BACnet dissector. (#1940)
• Added HAProxy protocol. (#2088)
• Added OICQ dissector. (#1950)
• Added OperaVPN detection
• ProtonVPN: add basic detection (#2006)
• Added detection of Facebook Reels and Stories
• Add an heuristic to detect fully encrypted flows (#2058)
• Added NDPI_MALWARE_HOST_CONTACTED flow risk
• Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk

Improvements

• Improve protocol detection for:
• FreeBSD compilation fix (C) update
• Gnutella: improve detection (#2019)
• H323: fix false positives (#1916)
• HTTP: fix another memory access error (#2049)
• HTTP: fix extraction of filename (#2046)
• HTTP: fix heap-buffer-overflow (#2044)
• HTTP: improve extraction of metadata and of flow risks (#1959)
• HTTP: remove useless code about XBOX (#1958)
• HTTP: rework state machine (#1966)
• Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
• Enhance DNS risk for long hostnames (> 32)
• Enhanced MS teams STUN/Azure detection
• Enhanced custom port definition and improved error reporting in case of duplications
• Improve detection of Alibaba flows (#1991)
• Improve detection of crawler/bot traffic (#1956)
• Improve detection of crawlers/bots (#1968)
• Improved MGCP detection by allowing '\r' as line feed.
• Improved MS Teams detection with heuristic
• Improved Steam detection by adding steamdiscover pattern. (#2105)
• Improved Wireguard detection
• Improved checks for duplicated entries in protocols file
• Improved classification further reducing memory used
• Improved detection of invalid chars in DNS names
• Improved domain search tet unit
• Improved helper scripts. (#1986)
• MS Teams enhancement
• MySql: improve detection (#1928)
• zabbix: improve detection (#2055)

Tools

• ndpiReader: allow to configure LRU caches TTL and size (#2004)
• ndpiReader: fix VXLAN de-tunneling (#1913)
• ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
• ndpiReader: fix export of HTTP attributes (#1982)
• ndpiReader: fix flow stats (#1943)
• ndpiReader: fix print of flow payload (#1960)
• ndpiReader: improve printing of payload statistics (#1989)
• ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
• ndpireader: fix detection of DoH traffic based on packet distributions (#2045)

Misc

• ARM compilation fix
• Add ndpi_domain_classify_finalize() function (#2084)
• Add a configuration knob to enable/disable loading of gambling list (#2047)
• Add a new flow risk about literal IP addresses used as SNI (#1892)
• Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
• Add another example of custom rules (#1923)
• Add support for multiline json
• Add support for roaring_bitmap_xor_inplace (#1983)
• Add support for vxlan decapsulation (#1441) (#1900)
• Added Source Engine dissector. (#1937)
• Added lists/gambling.list to extra dist.
• Added slackb.com SNI. (#2067)
• Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2)
• Added check to avoid skype heuristic false positives
• Added comment
• Added coverage targets to Makefile.am for convenience. (#2039)
• Added fix for better handling exceptions rollback in case of later match
• Added hyperlink
• Added ndpi_binary_bitmap data structure
• Added ndpi_bitmap64 support
• Added ndpi_bitmap_andnot API call
• Added ndpi_bitmap_copy() API call
• Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls
• Added ndpi_domain_classify_XXX(0 API
• Added ndpi_filter_add_multi() API call
• Added ndpi_murmur_hash to the nDPI API
• Added new API calls for implementing Bloom-filter like data structures
• Added printf/fprintf replacement for some internal modules. (#1974)
• Added scripts to auto generate hostname/SNI *.inc files. (#1984)
• Added sub-domain classification fix
• Added the ability to define custom protocols with arbitrary Ids in proto.txt
• Added vlan_id in ndpi_flow2json() prototype
• Adds new pcap for testing "funny" HTTP servers
• All protocols should be excluded sooner or later (#1969)
• Allow init of app protocols w/o any hostnames set. (#2057)
• Avoid calling ndpi_reconcile_protocols() twice in ndpi_detection_giveup() (#1996)
• Boundary check
• CI: fix Performance job (#1936)
• Centos7 fixes
• Changed logging callback function sig. (#2000)
• Changes for supporting more efficient sub-string matching
• Classification fixes
• DNS: extract geolocation information, if available (#2065)
• Debian 12 fixes
• Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted
• DisneyPlus/Hulu ip lists should be auto-generated (#1905)
• Extend content list of Microsoft protocols (#1930)
• Extend content-match list (#1967)
• Fix LRU/Patricia/Automa stats in ndpiReader with multiple threads (#1934)
• Fix MS Teams detection with heuristic (#1972)
• Fix access to packet/flow information (#2013)
• Fix an heap-buffer-overflow (#1994)
• Fix classification-by-ip in ndpi_detection_giveup (#1981)
• Fix compilation (#2011)
• Fix compilation in CI jobs (#2048)
• Fix compilation on Windows (#2072)
• Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
• Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)
• Fix export/serialization of flow->risk (#1885)
• Fix for buffer overflow in serialization
• Fix insert of ip addresses into patricia tree(s) (#1895)
• Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
• Fix packet counters (#1884)
• Fix some errors found by fuzzers (#2078)
• Fix some memory errors triggered by allocation failures (#1995)
• Fix some prototypes (#2085)
• Fix string truncation. (#2056)
• Fixed OpenWRT arm related build issues. (#2104)
• Fixed heap-buffer-overflow issue
• Fixed heap-overflow if compiled with --enable-tls-sigs. (#2038)
• Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
• Fixed missing AS_HELP_STRING in configure.ac. (#1893)
• Fixed two OpenWRT arm related build issues. (#2103)
• Fixes matches with domain name strings that start with a dot
• Fixes risk mask exception handling while improving the overall performance
• Implemented Count-Min Sketch [count how many times a value has been observed]
• Implemented Zoom/Teams stream type detection
• Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw
• Implemented ndpi_predict_linear() for predicting a timeseries value overtime
• Improved debug output. (#1951)
• Improved invalid logging via printf().
• Improved line protocol dissection with heuristic
• Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
• Improved protocol detection exploiting IP-based guess Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
• Improvement for reducing false positives
• Included Gambling website data from the Polish hazard.mf.gov.pl list (#2041)
• Keep master protocol in ndpi_reconcile_protocols
• Leak fix
• Language fix
• Line: fix heap-buffer-overflow error (#2015)
• Made VK protocol detection more strict
• Make Bittorrent LRU cache IPv6 aware. (#1909)
• Merged new and old version of ndpi_domain_classify.c code
• Mullvad VPN service added (based on entry node IP addresses) (#2062)
• Numeric truncation at ndpi_analyze.c at lines 101, 104, 107, 110 (#1999)
• Numeric truncation at tls.c:1010 (#2005)
• Ookla: rework detection (#1922)
• Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4()
• ProtonVPN: split the ip list (#2060)
• QUIC: add support for QUIC version 2
• QUIC: export QUIC version as metadata
• QUIC: fix a memory access error
• QUIC: fix dissection of packets forcing VN
• RDP: improve detection over UDP (#2043)
• RTP: remove dead-code (#1953)
• RTP: rework code (#2021)
• Refreshed ASN lists Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
• Remove some useless checks (#1993)
• Remove special handling of some TCP flows without SYN (#1965)
• Removed overlapping port
• Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI
• Replaces free() with ndpi_free()
• Rework CI jobs to try reducing CI duration (#1903)
• Reworked domain classification based on binary filters
• Reworked initialization
• Reworked ndpi_filter_xxx implementation using compressed bitmaps
• Reworked teams handling
• RiotGames: add detection of flows (#1935)
• STUN: add dissection of DTLS handshake (#2018)
• STUN: avoid FacebookVoip false positives (#2029)
• STUN: fix Skype/MsTeams detection and monitoring logic (#2028)
• STUN: fix detection of Google Voip apps (#2031)
• STUN: fix detection over TCP
• STUN: improve WhatsappCall detection
• STUN: keep monitoring/processing STUN flows (#2012)
• STUN: tell RTP from RTCP while in monitoring state (#2027)
• Serialization fix
• Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
• Simplify ndpi_internal_guess_undetected_protocol() (#1941)
• Simplify the report of streaming multimedia info (#2026)
• SoftEther: fix invalid memory access
• Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic approach for handling Internet domain names.
• Sync unit tests results
• Sync unit tests results
• Sync unit tests results (#1962)
• Sync utests results (#1887)
• TLS: add basic, basic, detection of Encrypted ClientHello (#2053)
• TLS: fix another interger overflow in certificate processing (#1915)
• TLS: fix parsing of certificate elements (#1910)
• Test files for riit games
• Test multiple ndpiReader configurations (#1931)
• Thrift: fix heap-buffer-overflow (#2024)
• Update GitHub runners versions (#1889)
• Update every ip lists (#2079)
• Update libinjection code (#1918)
• Update protocols documentation (#2081)
• Update roaring bitmap code
• Updated line test result
• Updated pcap detection results after Facebook Reel/Stories support
• Updated results
• Updated results after the latest changes
• Win include change
• Windows code rework
• Windows compilation fixes
• Windows warning checks
• add 2 ns from fdn.fr to DoH section (#1964)
• add support for gre decapsulation (#1442) (#1921)
• added bimap and/or with allocation
• added feature to extract filename from http attachment (#2037)
• added new domain names (#2002)
• configure: add an option to enable debug build, i.e -g (#1929)
• fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
• fixed numeric truncation error
• fixed numeric truncation error in diameter.c (#2034)
• fixed numeric truncation error in kerberos.c (#2032)
• fixed numeric truncation error in ndpi_main.c:6837 (#1998)
• fixed numeric truncation error in rtcp.c (#2033)
• fuzz: add a new fuzzer to test TLS certificates (#1901)
• fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926)
• fuzz: add fuzzer for DGA detection code (#2042)
• fuzz: add fuzzer to test internal gcrypt code (#1920)
• fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082)
• fuzz: add fuzzers to test reader_util code (#2080)
• fuzz: extend coverage (#2073)
• fuzz: extend fuzz coverage (#1888)
• fuzz: extend fuzzers coverage (#1952)
• fuzz: extend fuzzing coverage (#2040)
• fuzz: extend fuzzing coverage (#2052)
• fuzz: extend fuzzing coverage (#2083)
• fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
• fuzz: some improvements and add two new fuzzers (#1881)
• fuzzing: extend fuzzing coverage
• in case of failure, failing result files are not listed
• minor fixes (#2023)
• oss-fuzz: sync build script with upstream
• remove redefinition to vxlanhdr struct in vxlan dissector (#1911)
• removed useless call of ndpi_set_risk func (#2022)
• tests: add an option to force the overwrite of the unit tests results (#2001)
• tests: restore some old paths as symbolic links (#2050)
• tftp: check for Option Acknowledgements
• tftp: check incrementation for DATA and ACK packets
• tftp: rework request checking to account for options
• tftp: update pcap results
• version of dirent.c that is liked by both VC++ and MinGW

nDPI 4.6 (Feb 2023)

New Features

• New support for custom BPF protocol definition using nBPF (see example/protos.txt)
• Improved dissection performace
• Added fuzzing all over

New Supported Protocols and Services

• Add protocol detection for:
  • Activision
  • AliCloud server access
  • AVAST
  • CryNetwork
  • Discord
  • EDNS
  • Elasticsearch
  • FastCGI
  • Kismet
  • Line App and Line Voip valls
  • Meraki Cloud
  • Munin
  • NATPMP
  • Syncthing
  • TP-LINK Smart Home
  • TUYA LAN
  • SoftEther VPN
  • Tailscale
  • TiVoConnect

Improvements

• Improve protocol detection for:
  • Anydesk
  • Bittorrent (fix confidence, detection over TCP)
  • DNS, add ability to decode DNS PTR records used for reverse address resolution
  • DTLS (handle certificate fragments)
  • Facebook Voip calls
  • FastCGI (dissect PARAMS)
  • FortiClient (update default ports)
  • Zoom
    • Add Zoom screen share detection
    • Add detection of Zoom peer-to-peer flows in STUN
  • Hangout/Duo Voip calls detection, optimize lookups in the protocol tree
  • HTTP
    • Handling of HTTP-Proxy and HTTP-Connect
    • HTTP subclassification
    • Check for empty/missing user-agent in HTTP
  • IRC (credentials check)
  • Jabber/XMPP
  • Kerberos (support for Krb-Error messages)
  • LDAP
  • MGCP
  • MONGODB (avoid false positives)
  • Postgres
  • POP3
  • QUIC (support for 0-RTT packets received before the initial)
  • Snapchat Voip calls
  • SIP
  • SNMP
  • SMB (support for messages split into multiple TCP segments)
  • SMTP (support for X-ANONYMOUSTLS command)
  • STUN
  • SKYPE (improve detection over UDP, remove detection over TCP)
  • Teamspeak3 (License/Weblist detection)
  • Threema Messenger
  • TINC (avoid processing SYN packets)
  • TLS
    • improve reassembler
    • handling of ALPN(s) and subclassification
    • ignore invalid Content Type values
  • WindowsUpdate
• Add flow risk:
  • NDPI_HTTP_OBSOLETE_SERVER
  • NDPI_MINOR_ISSUES (generic/relevant information about issues found on traffic)
  • NDPI_HTTP_OBSOLETE_SERVER (Apache and nginx are supported)
  • NDPI_PERIODIC_FLOW (reserved bit to be used by apps based on nDPI)
  • NDPI_TCP_ISSUES
• Improve detection of WebShell and PHP code in HTTP URLs that is reported via flow risk
• Improve DGA detection
• Improve AES-NI check
• Improve nDPI JSON serialization
• Improve export/print of L4 protocol information
• Improve connection refused detection
• Add statistics for Patricia tree, Ahocarasick automa, LRU cache
• Add a generic (optional and configurable) expiration logic in LRU caches
• Add RTP stream type in flow metadata
• LRU cache is now IPv6 aware

Tools

• ndpiReader
  • Add support for Linux Cooked Capture v2
  • Fix packet dissection (CAPWAP and TSO)
  • Fix Discarded bytes statistics

Fixes

• Fix classification by-port
• Fix exclusion of DTLS protocol
• Fix undefined-behaviour in ahocorasick callback
• Fix infinite loop when a custom rule has port 65535
• Fix undefined-behavior when setting empty user-agent
• Fix infinite loop in DNS dissector (due to an integer overflow)
• Fix JSON export of IPv6 addresses
• Fix memory corruptions in Bittorrent, HTTP, SoftEther, Florensia, QUIC, IRC, TFTP dissectors
• Fix stop of extra dissection in HTTP, Bittorrent, Kerberos
• Fix signed integer overflow in ASN1/BER dissector
• Fix char/uchar bug in ahocorasick
• Fix endianess in IP-Port lookup
• Fix FastCGI memory allocation issue
• Fix metadata extraction in NAT-PMP
• Fix invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow)

Misc

• Support for Rocky Linux 9
• Enhance fuzzers to test nDPI configurations, memory allocation failures, serialization/deserialization, algorithms and data structures
• GitHub Actions: update to Node.js 16
• Size of LRU caches is now configurable

nDPI 4.4 (July 2022)

New Features

• Add risk information that describes why a specific risk was triggered also providing metadata
• Added API call ndpi_check_flow_risk_exceptions() for handling risk exceptions
• Split protocols in: network (e.g. TLS) and application protocols (e.g. Google)
• Extended confidence level with two new values (NDPI_CONFIDENCE_DPI_PARTIAL and NDPI_CONFIDENCE_DPI_PARTIAL_CACHE)
• Added ndpi_get_flow_error_code() API call

New Supported Protocols and Services

• Add protocol detection for:
  • UltraSurf
  • i3D
  • RiotGames
  • TSAN
  • TunnelBear VPN
  • collectd
  • PIM (Protocol Indipendent Multicast)
  • Pragmatic General Multicast (PGM)
  • RSH
  • GoTo products (mainly GoToMeeting)
  • Dazn
  • MPEG-DASH
  • Agora Software Defined Real-time Network (SD-RTN)
  • Toca Boca
  • VXLAN
  • MDNS/LLMNR

Improvements

• Improve protocol detection for:
  • SMTP/SMTPS now supports STARTTLS
  • OCSP
  • TargusDataspeed
  • Usenet
  • DTLS (added support for old versions)
  • TFTP
  • SOAP via HTTP
  • GenshinImpact
  • IPSec/ISAKMP
  • DNS
  • syslog
  • DHCP (various bug fixes and improvements)
  • NATS
  • Viber
  • Xiaomi
  • Raknet
  • gnutella
  • Kerberos
  • QUIC (Added support for v2drft 01)
  • SSDP
  • SNMP
• Improved DGA detection
• Improved AES-NI check
• Add flow risk:
  • NDPI_PUNYCODE_IDN
  • NDPI_ERROR_CODE_DETECTED
  • NDPI_HTTP_CRAWLER_BOT
  • NDPI_ANONYMOUS_SUBSCRIBER
• NDPI_UNIDIRECTIONAL_TRAFFIC

Changes

• Added support for 64 bit bins
• Added Cloudflare WARP detection patterns
• Renamed Z39.50 -> Z3950
• Replaced nDPI's internal hashmap with uthash
• Reimplemented 1kxun application protoco
• Renamed SkypeCall to Skype_TeamsCall
• Updated Python Bindings
• Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation

Fixes

• Fixes for some protocol classification families
• Fixed default protocol ports for email protocols
• Various memory and overflow fixes
• Disabled various risks for specific protocols (e.g. disable missing ALPN for CiscoVPN)
• Fix TZSP decapsulation

Misc

• Update ASN/IPs lists
• Improved code profiling
• Use Doxygen to generate the API documentation
• Added Edgecast and Cachefly CDNs.

Raw Changelog

• Label SMTP w/ STARTTLS as SMTPS and dissect TLS clho. (#1639)
• Compilation fix
• Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636)
• SMTP with STARTTLS is now identified as SMTPS
• Detect SMTPs w/ STARTTLS as TLS and dissect client/server hello. Fixes #1630. (#1637)
• Run regression tests from different locations at the same time w/o side effects on the results. (#1638)
• Exported username in flow information
• Updated ndpi_check_flow_risk_exceptions() signature
• Cleaned-up issuer DN check code adding u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str, char *issuerDN);
• Set CiscoVPN as a network protocol
• Updated JA3/SSL fingerprints.
• Replaced malicious JA3-md5/SSL-cert-sha1 ac automata with hashmaps.
• Added UltraSurf protocol dissector. (#1618)
• Add two new confidence values: confidence by partial DPI (#1632)
• Update host content list match (#1633)
• Sync Psiphon unit test. (#1634)
• Added Psiphon detection patterns. See #566 and #1099. (#1631)
• OCSP: improve detection (#1629)
• Added i3D and RiotGames protocol dissectors. (#1609)
• TargusDataspeed: avoid false positives (#1628)
• Update ASN/IPs lists (#1627)
• bins: add support for 64bit bins (#1626)
• Skinny: rework and improve classification (#1625)
• Skype_Teams, Mining, SnapchatCall: fix flow category (#1624)
• Minor changes in how classification results are set (#1623)
• Usenet: improve dissection (#1622)
• Fix category for mail sessions (#1621)
• TLS: add support for old DTLS versions and for detection of mid-sessions (#1619)
• Fix a compilation warning (#1620)
• Generate profiling results as PNG.
• gprof test/CI integration
• Improved TFTP. Dissect Read/Write Request filenames. (#1617)
• Added TSAN support. (#1613)
• Fix byte-order issue during ndpiReader tcp/udp src/dst port serialization. Fixes #1608. (#1614)
• Added Cloudflare WARP detection patterns. (#1615) (#1616)
• Fixed SMTP default port 587
• Added TunnelBear VPN detection patterns. (#1615)
• Updated (C)
• Removed space from "Genshin Impact"
• sync unit tests (#1612)
• Fix after the protocol name update
• Renamed Z39.50 -> Z3950 as the '.' breaks the naming convention QUIC is a network protocol
• Enhanced TLS risk info reported to users
• Added default port for syslog TCP
• Fix compilation and sync unit tests results (#1606)
• Added unidirectional traffic flow risk
• Improved SOAP via HTTP. (#1605)
• Improved GenshinImpact protocol dissector. (#1604)
• Added collectd dissector (again). (#1601)
• Replaced nDPI's internal hashmap with uthash. (#1602)
• Improved IPSec/ISAKMP detection. (#1600)
• Added new test pcaps
• Add some statistics to ndpiReader (#1587)
• Add support for PIM (Protocol Indipendent Multicast) protocol (#1599)
• Improved WhatsApp detection. (#1595)
• Fix invalid memory access (#1596)
• DNS: fix TTL check and sync unit test results (#1594)
• Updated DNS alert triggered only with TTL == 0
• Restored ndpi_set_proto_defaults() prototype Updated test results
• Added check for DGA names that resolve to a valid record
• Improved DNS traffic analysis Added ability to identify application and network protocols
• Added DNS record TTL check
• Added gprof CPU/HEAP profiling support. (#1592)
• Removed Makefile references to legacy code. (#1589)
• Added Pragmatic General Multicast (PGM) protocol detection
• Dissect host line if SSDP contains such. (#1586)
• Reimplemented 1kxun application protocol. (#1585)
• Prevent compilation failure if, for whatever reason, NDPI_API_VERSION is empty. (#1584)
• Fixed syslog false negatives. (#1582)
• Fix some debug messages (#1583)
• Updated test results
• Fixed invalid DHCP dissection
• Fixed DHCP dissection bug
• Added RSH dissector. Fixes #202. (#1581)
• Add support for GoTo products (mainly GoToMeeting) (#1580)
• Fix syslog heap overflow introduced in 09fbe0a64a11b08a35435f516e9a19f7e0c20d7c. (#1579)
• Fixed syslog false positives. (#1577)
• Fix heap buffer overflow mentioned in #1574. (#1576)
• TLS: fix use-of-uninitialized-value error (#1573)
• Removed README.nDPI as it does not provide any new information not covered by README.md (#1572)
• Removed LGTM ql query for packet payload integer arithmetic. (#1570)
• Force roaring bitmap to use ndpi memory wrappers. (#1569)
• TLS: fix stack-buffer-overflow error (#1567)
• Updated risk results
• Improved message for known proto on non std port
• Added check
• Updated README.md (#1562)
• TLS: fix use-of-uninitialized-value error (#1566)
• Redefined type name to avoid conflicts
• Added ability to return risk info in JSON format in ndpi_get_flow_risk_info()
• Support word diff for tests/do.sh for better readability. (#1565)
• Prohibit MPEG-DASH to set HTTP as application protocol. (#1560)
• HTTP: fix heap-buffer-overflow error (#1564)
• Certificate timestamps should be printed in UTC (#1563)
• Fixed dispay bug for risk_info
• Updated tests results Code cleanup
• Added RiskInfo string
• Fix dissection of IPv4 header (#1561)
• Dazn: add support for Dazn streaming service (#1559)
• Compilation fixes for old ggc's
• Comment
• Added detection for WordPress exploits Fixed ndpi_iph_is_valid_and_not_fragmented() that was bugged with non UDP traffic
• Use Doxygen to generate the API documentation. (#1558)
• Added MPEG-DASH dissector. Fixes #1223. (#1555)
• Fixed HTTP lower/upper protocol mess for Aimini/IPP. (#1557)
• Compilation fixes for old gcc compiler
• Compilation fixes
• Version cut fix
• Fixes compilation issues on RedHat systems
• Sync unit test results (#1554)
• Updated SkypeCall -> Skype_TeamsCall
• Fixed false positives with NATS
• Added script to compare and verify the output of `make dist'. (#1551)
• Replaced obsolete autoconf macros. (#1553)
• Fixed windows-latest build error. (#1552)
• Improved invalid host detection
• Added invalid SNI check in QUIC
• Improved detection of invalid SNI and hostnames in TLS, HTTP
• Added room for storing information used by custom third-party dissectors
• Moved RTSP http patterns to the protocol source file.
• Yet another approach to fix #1499 (basically a copy&pasta from @socketpair).
• Removed MacOS XCode integration.
• Moved mgcp.pcapng to tests/pcap/ instead of tests/
• DNS-over-QUIC: update default port (#1548)
• Improved Viber (TCP) detection. (#1547)
• Improved Xiaomi HTTP detection. (#1546)
• Removed TLS patterns in the CiscoVPN aka Anyconnect dissector as mentioned in PR #1534. (#1543)
• Added Softether(-VPN) DDNS service detection. (#1544)
• Improved TLS alert detection. (#1542)
• Improved TLS application data detection. (#1541)
• Added Edgecast and Cachefly CDNs. (#1540)
• Replaced ndpiReader's libjson-c support with libnDPI's internal serialization interface. (#1535)
• Fix compilation (if --enable-debug-messages is used) (#1539)
• Added extra check to make sure that the guessed protocol is the one we expect and not another one
• Fixes bug that prevents triggering alerts for traffic on non-standard ports that have been defined in the custom protocols file
• Fixes outdated description
• Modified risk labels
• Added some Pluralsight Hostnames/SNIs. May fix #1501. (#1538)
• Updated RRD dependencies
• Improved suspicious http user agent detection. (#1537)
• Added ndpi_get_flow_error_code() API call Fixed typo
• Improved AES-NI check. (#1536)
• Improved AES-NI check on Linux to avoid crashes on CPUs that do not support it (e.g. Intel Celeron N2930)
• Sync unit tests results (#1533)
• Improved TLS application data detection. (#1532)
• Added BPF filtering for discarding non-IP packets
• String messages have been shrinked
• Added ability to store custom category file in patricia tree
• Add ndpi_json_string_escape to the API
• Raknet: fix heap-buffer-overflow (#1531)
• Added generic user agent setter. (#1530)
• XIAOMI: add detection of Xiaomi traffic (#1529)
• Added RakNet protocol dissector. (#1527)
• Code cleanup (removed redundancy)
• Tiny gnutella improvement if gtk-gnutella used. (#1525)
• Updated `utils/whatsapp_ip_addresses_download.sh' to scrape the required IP addresses/ranges. (#1524)
• Add some scripts to easily update some IPs lists (#1522)
• Reduce ndpiReader's -h' spam. -H' does this job now. (#1523)
• Added proprietary Agora Software Defined Real-time Network (SD-RTN) protocol dissector. (#1520)
• Added Toca Boca protocol dissector. (#1517)
• Removed superfluous ifdef'd includes. (#1519)
• Improved sflow protocol detection false-positives. (#1518)
• Kerberos: fix Undefined-shift error (#1516)
• DGA improvements
• Minor fix.
• Merge pull request #1491 from utoni/fix/windows-msys2
• Fixed msys2 build warnings and re-activated CI Mingw64 build.
• Kerberos: fix some memory access errors (#1514)
• Extended list of cybersecurity domains
• fix(ndpi_main):Fix memory leak about ndpi_str; (#1513)
• TINC: fix invalid memory read (#1512)
• Improved ASN.1 parsing for Keberos. Fixes #1492. (#1497)
• QUIC: handle retransmissions and overlapping fragments in reassembler (#1195) (#1498)
• Fix JSON-C.
• Python bindings fix.
• Added ndpi_find_outliers() API call using Z-Score
• Added -z flag
• ndpiReader: fix compilation (#1510)
• Removed un-necessary guess in mining
• update
• Fixed incompatibilities due to https://github.com/ntop/nDPI/pull/1509
• DGA improvements
• Waring fixes
• ndpireader: add json output back. (#1509)
• Improvements for CUSTOM_NDPI_PROTOCOLS
• Moved geneated file to a separate folder
• Improved twitter detection
• Removed SRV record from suspicious DNS traffic
• Improved DGA detection
• [autoconf] Fixed .git submodule detection test. (#1507)
• Added code for identifiying anomalies with metrics stored in InfluxDB
• reader_util: add support for userAgent in SSDP (#1502)
• Add support for Pluralsight site (#1503)
• Fix CI tests results (#1504)
• Reducing the size of the ndpi_detection_module_struct structure. (#1490)
• [SSDP] Extract HTTP user-agent when available. (#1500)
• Improved DGA detection skipping names containign at least 3 consecutive digits in the first word
• QUIC: add support for version 2 draft 01 (#1493)
• Mining: cleanup registration (#1496)
• Trying to improve QUIC reassembler (#1195) (#1489)
• Update Python bindings guide.
• Fix typo.
• Add HOWTO Python.
• Fix python bindings CI.
• Complete rework of nDPI Python bindings (cffi API, automatic generation, packaging and CI integration)
• Extended the list of cybersecurity protocol
• Bug fixing. (#1487)
• QUIC: convert logs to standard mechanism (#1485)
• QUIC: fix dissection of draft-34 (#1484)
• Extend tests coverage (#1476)
• Improved ASN/IP update scripts and CI integration. (#1474)
• Implement CI on Windows. (#1483)
• Some small fixes (#1481)
• Extracting the Azure Origin url from the download link (#1480)
• Errors fixed (#1482)
• EthernetIP: fix integer conversion on big-endian archs (#1477)
• Fixed a bug for BE architectures (#1478)
• configure: fix usage of libgpg-error with --with-local-libgcrypt (#1472)
• Added autoconf option `--enable-tls-sigs'. (#1471)
• Drop support for non-gcrypt builds. (#1469)
• Internal crypto: increase size of authentication buffer (#1468)
• reader_util: fix parsing of MPLS packets (#1467)
• Add ICMP checksum check and set risk if mismatch detected. (#1464)
• Typo
• Added configureable ndpi packet processing limit. (#1466)
• Fix libgcrypt(-light/-internal) compile error. (#1465)
• Add a new flow risk NDPI_ANONYMOUS_SUBSCRIBER (#1462)
• Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation
• Removed some unused fields (#1461)
• Bug fixing. (#1459)
• Added `--enable-code-coverage' build using lcov for coverage generation. (#1430)
• reader_util: fix TZSP decapsulation (#1460)
• Add some scripts to easily update some IPs lists (#1449)
• Provide some API functions for convenience. (#1456)
• Win fixes
• Replaced strdup with ndpi_strup
• Directly drop malformed packets (#1455)
• reader_util: fix parsing of IPv6 extension headers (#1453)
• reader_util: fix infinite loop in packet dissection (#1454)
• fuzz: purge old sessions (#1451)
• DTLS: fix access to certificate cache (#1450)
• EthernetIP: add missing initialization (#1448)
• Add support for Google Cloud (#1447)
• fuzz: make fuzz_ndpi_reader faster (#1446)
• Added lightweight implementation of libgcrypt. (#1444)
• Fix compilation and sync unit tests results (#1445)
• Added newflow risk NDPI_HTTP_CRAWLER_BOT
• Silenced NDPI_SUSPICIOUS_DGA_DOMAIN, NDPI_BINARY_APPLICATION_TRANSFER, NDPI_HTTP_NUMERIC_IP_HOST, NDPI_MALICIOUS_JA3,
• Extended cybersecurity protocol dissection
• Added SNMP error code check
• Exteended cybersecurity list
• Invalid prototupe fix
• HSRP: fix dissection over IPv6 (#1443)
• Added cybersecurity category mapping to string
• Added cybersecurity protocol and category that groups traffic towards leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways
• HSRP: add support for IPv6 (#1440)
• Added VXLAN dissector (#1439)
• Fix memory access in ndpi_strnstr() (#1438)
• Add few scripts to easily update some IPs lists (#1436)
• Increment current/total number of active flows on successful flow insertion (#1434)
• Added ndpi_serialize_string_string_len() APi call Fixed CSV string serialization
• Added HSRP protocol detection Removed attic directory now obsolete
• Added check to ignore multicast packets marking the as Skype
• Improved MDNS/LLMNR detection. (#1437)
• TLS: fix parsing of certificate elements (#1435)
• Sync utests (#1433)
• Add comment
• Updated test results
• Added NDPI_ERROR_CODE_DETECTED risk
• Renamed DCERPC to more generic RPC protocol so we can use also for other types of RPCs (not limited to DCE) Extended HTTP plugin to support RPC Improved HTTP crear text detection to limit it to Basic and Digest
• Typo
• Improved risks description
• Updated risk documentation
• Added new IDN/Punycode risk for spotting internationalized domain names
• Added missing __sync_fetch_and_add() definition in Windows

nDPI 4.2 (Feb 2022)

New Features

• Add a "confidence" field indicating the reliability of the classification
• Add risk exceptions for services and domain names via ndpi_add_domain_risk_exceptions()
• Add ability to report whether a protocol is encrypted

New Supported Protocols and Services

• Add protocol detection for:
  • Badoo
  • Cassandra
  • EthernetIP

Improvements

• Significantly reduced memory footprint from 2.94 KB to 688 B per flow
• Improve protocol detection for:
  • BitTorrent
  • ICloud Private Relay
  • IMAP, POP3, SMTP
  • Log4J/Log4Shell
  • Microsoft Azure
  • Pandora TV
  • RTP
  • RTSP
  • Salesforce
  • STUN
  • Whatsapp
  • QUICv2
  • Zoom
• Add flow risk:
  • NDPI_CLEAR_TEXT_CREDENTIALS
  • NDPI_POSSIBLE_EXPLOIT (Log4J)
  • NDPI_TLS_FATAL_ALERT
  • NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
• Update WhatsAPP and Instagram addresses
• Update the list of default ports for QUIC
• Update WindowsUpdate URLs
• Add support for the .goog Google TLD
• Add googletagmanager.com
• Add bitmaps and API for handling compressed bitmaps
• Add JA3 in risk exceptions
• Add entropy calculation to check for suspicious (encrypted) payload
• Add extraction of hostname in SMTP
• Add RDP over UDP dissection
• Add support for TLS over IPV6 in Subject Alt Names field
• Improve JSON and CSV serialization
• Improve IPv6 support for almost all dissectors
• Improve CI and unit tests, add arm64, armhf and s390x as part of CI
• Improve WHOIS detection, reduce false positives
• Improve DGA detection for skipping potential DGAs of known/popular domain names
• Improve user agent analysis
• Reworked HTTP protocol dissection including HTTP proxy and HTTP connect

Changes

• TLS obsolete protocol is set when TLS < 1.2 (used to be 1.1)
• Numeric IPs are not considered for DGA checks
• Differentiate between standard Amazon stuff (i.e market) and AWS
• Remove Playstation VUE protocol
• Remove pandora.tv from Pandora protocol
• Remove outdated SoulSeek dissector

Fixes

• Fix race conditions
• Fix dissectors to be big-endian friendly
• Fix heap overflow in realloc wrapper
• Fix errors in Kerberos, TLS, H323, Netbios, CSGO, Bittorrent
• Fix wrong tuple comparison
• Fix ndpi_serialize_string_int64
• Fix Grease values parsing
• Fix certificate mismatch check
• Fix null-dereference read for Zattoo with IPv6
• Fix dissectors initialization for XBox, Diameter
• Fix confidence for STUN classifications
• Fix FreeBSD support
• Fix old GQUIC versions on big-endian machines
• Fix aho-corasick on big-endian machines
• Fix DGA false positive
• Fix integer overflow for QUIC
• Fix HTTP false positives
• Fix SonarCloud-CI support
• Fix clashes setting the hostname on similar protocols (FTP, SMTP)
• Fix some invalid TLS guesses
• Fix crash on ARM (Raspberry)
• Fix DNS (including fragmented DNS) dissection
• Fix parsing of IPv6 packets with extension headers
• Fix extraction of Realm attribute in STUN
• Fix support for START-TLS sessions in FTP
• Fix TCP retransmissions for multiple dissectors
• Fix DES initialisation
• Fix Git protocol dissection
• Fix certificate mismatch for TLS flows with no client hello observed
• Fix old versions of GQUIC on big-endian machines

Misc

• Add tool for generating automatically the Azure IP list

New Features

• Add API for computing RSI (Relative Strenght Index)
• Add GeoIP support
• Add fragments management
• Add API for jitter calculation
• Add single exponential smoothing API
• Add timeseries forecasting support implementing Holt-Winters with confidence interval
• Add support for MAC to radix tree and expose the full API to applications
• Add JA3+, with ALPN and elliptic curve
• Add double exponential smoothing implementation
• Extended API for managing flow risks
• Add flow risk score
• New flow risks:
  • Desktop or File Sharing Session
  • HTTP suspicious content (useful for tracking trickbot)
  • Malicious JA3
  • Malicious SHA1
  • Risky domain
  • Risky AS
  • TLS Certificate Validity Too Long
  • TLS Suspicious Extension

New Supported Protocols and Services

• New protocols:
  • AmongUs
  • AVAST SecureDNS
  • CPHA (CheckPoint High Availability Protocol)
  • DisneyPlus
  • DTLS
  • Genshin Impact
  • HP Virtual Machine Group Management (hpvirtgrp)
  • Mongodb
  • Pinterest
  • Reddit
  • Snapchat VoIP calls
  • Tumblr
  • Virtual Asssitant (Alexa, Siri)
  • Z39.50
• Add protocols to HTTP as subprotocols
• Add detection of TLS browser type
• Add connectionless DCE/RPC detection

Improvements

• 2.5x speed bump. Example ndpiReader with a long mixed pcap v3.4 - nDPI throughput: 1.29 M pps / 3.35 Gb/sec v4.0 - nDPI throughput: 3.35 M pps / 8.68 Gb/sec
• Improve detection/dissection of:
• AnyDesk
• DNS
• Hulu
• DCE/RPC (avoid false positives)
• dnscrypt
• Facebook (add new networks)
• Fortigate
• FTP Control
• HTTP
  • Fix user-agent parsing
  • Fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined
• IEC104
• IEC60870
• IRC
• Netbios
• Netflix
• Ookla speedtest (detection over IPv6)
• openspeedtest.com
• Outlook / MicrosoftMail
• QUIC
  • update to draft-33
  • improve handling of SNI
  • support for fragmented Client Hello
  • support for DNS-over-QUIC
• RTSP
• RTSP via HTTP
• SNMP (reimplemented)
• Skype
• SSH
• Steam (Steam Datagram Relay - SDR)
• STUN (avoid false positives, improved Skype detection)
• TeamViewer (add new hosts)
• TOR (update hosts)
• TLS
  • Certificate Subject matching
  • Check for common ALPNs
  • Reworked fingerprint calculation
  • Fix extraction for TLS signature algorithms
  • Fix ClientHello parsing
• UPnP
• wireguard
• Improve DGA detection
• Improve JA3
• Improve Mining detection
• Improve string matching algorithm
• Improve ndpi_pref_enable_tls_block_dissection
• Optimize speed and memory size
• Update ahocorasick library
• Improve subprotocols detection

Fixes

• Fix partial application matching
• Fix multiple segfault and leaks
• Fix uninitialized memory use
• Fix release of patterns allocated in ndpi_add_string_to_automa
• Fix return value of ndpi_match_string_subprotocol
• Fix setting of flow risks on 32 bit machines
• Fix TLS certificate threshold
• Fix a memory error in TLS JA3 code
• Fix false positives in Z39.50
• Fix off-by-one memory error for TLS-JA3
• Fix bug in ndpi_lru_find_cache
• Fix invalid xbox and playstation port guesses
• Fix CAPWAP tunnel decoding
• Fix parsing of DLT_PPP datalink type
• Fix dissection of QUIC initial packets coalesced with 0-RTT one
• Fix parsing of GTP headers
• Add bitmap boundary checks

Misc

• Update download category name
• Update category labels
• Renamed Skype in Skype_Teams (the protocol is now shared across these apps)
• Add IEC analysis wireshark plugin
• Flow risk visualization in Wireshark
• ndpiReader
  • add statistics about nDPI performance
  • fix memory leak
  • fix collecting of risks statistics
• Move installed libraries from /usr/local to /usr
• Improve NDPI_API_VERSION generation
• Update ndpi_ptree_match_addr prototype

New Features

• Completely reworked and extended QUIC dissector
• Added flow risk concept to move nDPI towards result interpretation
• Added ndpi_dpi2json() API call
• Added DGA risk for names that look like a DGA
• Added HyperLogLog cardinality estimator API calls
• Added ndpi_bin_XXX API calls to handle bin handling
• Fully fuzzy tested code that has greatly improved reliability and robustness

New Supported Protocols and Services

• QUIC
• SMBv1
• WebSocket
• TLS: added ESNI support
• SOAP
• DNScrypt

Improvements

• Python CFFI bindings
• Various TLS extensions and fixes including extendede metadata support
• Added various pcap files for testing corner cases in protocols
• Various improvements in JSON/Binary data serialization
• CiscoVPN
• H323
• MDNS
• MySQL 8
• IEC 60870-5-104
• DoH/DoT dissection improvements
• Office365 renamed to Microsoft365
• Major protocol dissection improvement in particular with unknwon traffic
• Improvement in Telegram v6 protocol support
• HTTP improvements to detect file download/upload and binary files
• BitTorrent and WhatsApp dissection improvement
• Spotify
• Added detection of malformed packets
• Fuzzy testing support has been greatly improved
• SSH code cleanup

Fixes

• Fixed various memory leaks and race conditions in protocol decoding
• NATS, CAPWAP dissector
• Removed HyperScan support that greatly simplified the code
• ARM platform fixes on memory alignment
• Wireshark extcap support
• DPDK support
• OpenWRT, OpenBSD support
• MINGW compiler support

MISC

• Created demo app for nDPI newcomers
• Removed obsolete pplive and pando protocols

New Features

• New API calls
• Protocol detection: ndpi_is_protocol_detected
• Categories: ndpi_load_categories_file / ndpi_load_category
• JSON/TLV serialization: ndpi_serialize_string_boolean / ndpi_serialize_uint32_boolean
• Patricia tree: ndpi_load_ipv4_ptree
• Module initialization: ndpi_init_detection_module / ndpi_finalize_initalization
• Base64 encoding: ndpi_base64_encode
• JSON export: ndpi_flow2json
• Print protocol: ndpi_get_l4_proto_name / ndpi_get_l4_proto_info
• Libfuzz integration
• Implemented Community ID hash (API call ndpi_flowv6_flow_hash and ndpi_flowv4_flow_hash)
• Detection of RCE in HTTP GET requests via PCRE
• Integration of the libinjection library to detect SQL injections and XSS type attacks in HTTP requests

New Supported Protocols and Services

• TLS: new decode
• Added ALPN support
• Added export of supported version in TLS header
• Added Telnet dissector with metadata extraction
• Added Zabbix dissector
• Added POP3/IMAP metadata extraction
• Added FTP user/password extraction
• Added NetBIOS metadata extraction
• Added Kerberos metadata extraction
• Implemented SQL Injection and XSS attack detection
• Host-based detection improvements and changes
• Added Microsoft range
• Added twitch.tv website
• Added brasilbandalarga.com.br and .eaqbr.com.br as EAQ
• Added 20.180.0.0/14, 20.184.0.0/13 range as Skype
• Added 52.84.0.0/14 range as Amazon
• Added pastebin.com
• Changed 13.64.0.0/11 range from Skype to Microsoft
• Refreshed Whatsapp server list, added whatsapp-.fbcdn.net IPs
• Added public DNSoverHTTPS servers

Improvements

• Reworked and improved the TLS dissector
• Reworked Kerberos dissector
• Improved DNS response decoding
• Support for DNS continuous flow dissection
• Improved Python bindings
• Improved Ethereum support
• Improved categories detection with streaming and HTTP
• Support for IP-based detection to compute the application protocol
• Renamed protocol 104 to IEC60870 (more meaningful)
• Added failed authentication support with FTP
• Renamed DNSoverHTTPS to handle bot DoH and DoT
• Implemented stacked DPI decoding
• Improvements for CapWAP and Bloomberg
• Improved SMB dissection
• Improved SSH dissection
• Added capwap support
• Modified API signatures for ndpi_ssl_version2str / ndpi_detection_giveup
• Removed ndpi_pref_http_dont_dissect_response / ndpi_pref_dns_dont_dissect_response (replaced by ndpi_extra_dissection_possible)

Fixes

• Fixed memory invalid access in SMTP and leaks in TLS
• Fixed a few memory leaks
• Fixed invalid memory access in a few protocol dissectors (HTTP, memcached, Citrix, STUN, DNS, Amazon Video, TLS, Viber)
• Fixed IPv6 address format across the various platforms/distributions
• Fixed infinite loop in ndpi_workflow_process_packet
• Fixed SHA1 certificate detection
• Fixed custom protocol detection
• Fixed SMTP dissection (including email)
• Fixed Telnet dissection and invalid password report
• Fixed invalid category matching in HTTP
• Fixed Skype and STUN false positives
• Fixed SQL Injection detection
• Fixed invalid SMBv1 detection
• Fixed SSH dissection
• Fixed ndpi_ssl_version2str
• Fixed ndpi_extra_dissection_possible
• Fixed out of bounds read in ndpi_match_custom_category

Misc

ndpiReader

• CSV output enhancements
• Added tunnelling decapsulation
• Improved HTTP reporting
• Added scan and HTTP attacks (XSS, SQL Injection) detection

New Features

• nDPI now reports the protocol ASAP even when specific fields have not yet been dissected because such packets have not yet been observed. This is important for inline applications that can immediately act on traffic. Applications that need full dissection need to call the new API function ndpi_extra_dissection_possible() to check if metadata dissection has been completely performed or if there is more to read before declaring it completed.
• TLS (formerly identified as SSL in nDPI v2.x) is now dissected more deeply, certificate validity is extracted as well certificate SHA-1.
• nDPIreader can now export data in CSV format with option -C
• Implemented Sequence of Packet Length and Time (SPLT) and Byte Distribution (BD) as specified by Cisco Joy (https://github.com/cisco/joy). This allows malware activities on encrypted TLS streams. Read more at https://blogs.cisco.com/security/detecting-encrypted-malware-traffic-without-decryption
  • Available as library and in ndpiReader with option -J
• Promoted usage of protocol categories rather than protocol identifiers in order to classify protocols. This allows application protocols to be clustered in families and thus better managed by users/developers rather than using hundred of protocols unknown to most of the people.
• Added Inter-Arrival Time (IAT) calculation used to detect protocol misbehaviour (e.g. slow-DoS detection)
• Added data analysis features for computign metrics such as entropy, average, stddev, variance on a single and consistent place that will prevent when possible. This should ease traffic analysis on monitoring/security applications. New API calls have been implemented such as ndpi_data_XXX() to handle these calculations.
• Initial release of Python bindings available under nDPI/python.
• Implemented search of human readable strings for promoting data exfiltration detection
  • Available as library and in ndpiReader with option -e
• Fingerprints
  • JA3 (https://github.com/salesforce/ja3)
  • HASSH (https://github.com/salesforce/hassh)
  • DHCP
• Implemented a library to serialize/deserialize data in both Type-Length-Value (TLV) and JSON format
  • Used by nProbe/ntopng to exchange data via ZMQ

New Supported Protocols and Services

• DTLS (i.e. TLS over UDP)
• Hulu
• TikTok/Musical.ly
• WhatsApp Video
• DNSoverHTTPS
• Datasaver
• Line protocol
• Google Duo and Hangout merged
• WireGuard VPN
• IMO
• Zoom.us

Improvements

• TLS
  • Organizations
  • Ciphers
  • Certificate analysis
• Added PUBLISH/SUBSCRIBE methods to SIP
• Implemented STUN cache to enhance matching of STUN-based protocols
• Dissection improvements
  • Viber
  • WhatsApp
  • AmazonVideo
  • SnapChat
  • FTP
  • QUIC
  • OpenVPN support for UDP-based VPNs
  • Facebook Messenger mobile
  • Various improvements for STUN, Hangout and Duo
• Added new categories: CUSTOM_CATEGORY_ANTIMALWARE, NDPI_PROTOCOL_CATEGORY_MUSIC, NDPI_PROTOCOL_CATEGORY_VIDEO, NDPI_PROTOCOL_CATEGORY_SHOPPING, NDPI_PROTOCOL_CATEGORY_PRODUCTIVITY and NDPI_PROTOCOL_CATEGORY_FILE_SHARING
• Added NDPI_PROTOCOL_DANGEROUS classification

Fixes

• Fixed the dissection of certain invalid DNS responses
• Fixed Spotify dissection
• Fixed false positives with FTP and FTP_DATA
• Fix to discard STUN over TCP flows
• Fixed MySQL dissector
• Fix category detection due to missing initialization
• Fix DNS rsp_addr missing in some tiny responses
• Various hardening fixes

New Supported Protocols and Services

• Added Modbus over TCP dissector

Improvements

• Wireshark Lua plugin compatibility with Wireshark 3
• Improved MDNS dissection
• Improved HTTP response code handling
• Full dissection of HTTP responses

Fixes

• Fixed false positive mining detection
• Fixed invalid TCP DNS dissection
• Releasing buffers upon realloc failures
• ndpiReader: Prevents references after free
• Endianness fixes
• Fixed IPv6 HTTP traffic dissection
• Fixed H.323 detection

Other

• Disabled ookla statistics which need to be improved
• Support for custom protocol files of arbitrary length
• Update radius.c to RFC2865

New Supported Protocols and Services

• New Bitcoin, Ethereum, ZCash, Monero dissectors all identified as Mining
• New Signal.org dissector
• New Nest Log Sink dissector
• New UPnP dissector
• Added support for SMBv1 traffic, split from SMBv23

Improvements

• Improved Skype detection, merged Skype call in/out into Skype Call
• Improved heuristics for Skype, Teredo, Netbios
• Improved SpeedTest (Ookla) detection
• Improved WhatsApp detection
• Improved WeChat detection
• Improved Facebook Messenger detection
• Improved Messenger/Hangout detection
• Improved SSL detection, prevent false positives
• Improved guess for UDP protocols
• Improved STUN detection
• Better Hyperscan integration
• Added more Ubuntu servers
• Added missing categorization with giveup/guess
• Optimisations for TCP flows that do not start with a SYN packet (early giveup)

Fixes

• Fixed eDonkey false positives
• Fixed Dropbox dissector
• Fixed Spotify dissector
• Fixed custom protocol loading
• Fixed missing Application Data packet for TLS
• Fixed buffer overflows
• Fixed custom categories match by IP
• Fixed category field not accounted in ndpi_get_proto_category
• Fixed null pointer dereference in ndpi_detection_process_packet
• Fixed compilation on Mac

Other

• Deb and RPM packages: ndpi with shared libraries and binaries, ndpi-dev with headers and static libraries
• Protocols now have an optional subprotocol: Spotify cannot have subprotocols, DNS can (DNS.Spotify)

New API functions:

• ndpi_fill_ip_protocol_category() to handle ICMP flows category
• ndpi_flowv4_flow_hash() and ndpi_flowv6_flow_hash() to support the Community ID Flow Hashing (https://github.com/corelight/community-id-spec)
• ndpi_protocol2id() to print the protocol as ID
• ndpi_get_custom_category_match() to search host in custom categories
• Changed ndpi_detection_giveup() API: guess is now part of the call
• Added DPDK support to ndpiReader
• Removed Musical.ly protocol (service no longer used)
• Custom categories have now priority over protocol related categories
• Improved clang support