Capa Versions Save

The FLARE team's open-source tool to identify capabilities in executable files.

v7.0.1

2 months ago

This release fixes a circular import error when using capa as a library.

Bug Fixes

  • fix potentially circular import errors #1969 @williballenthin

Raw diffs

v7.0.0

2 months ago

This is the v7.0.0 release of capa which was mainly worked on during the Google Summer of Code (GSoC) 2023. A huge shoutout to our GSoC contributors @colton-gabertan and @yelhamer for their amazing work. See our blog posts for more details:

Also, a big thanks to the other contributors: @aaronatp, @Aayush-Goel-04, @bkojusner, @doomedraven, @ruppde, @larchchen, @JCoonradt, and @xusheng6.

New Features

  • add Ghidra backend #1770 #1767 @colton-gabertan @mike-hunhoff
  • add Ghidra UI integration #1734 @colton-gabertan @mike-hunhoff
  • add dynamic analysis via CAPE sandbox reports #48 #1535 @yelhamer
    • add call scope #771 @yelhamer
    • add thread scope #1517 @yelhamer
    • add process scope #1517 @yelhamer
    • rules: change meta.scope to meta.scopes @yelhamer
    • protobuf: add Metadata.flavor @williballenthin
  • binja: add support for forwarded exports #1646 @xusheng6
  • binja: add support for symtab names #1504 @xusheng6
  • add com class/interface features #322 @Aayush-goel-04
  • dotnet: emit enclosing class information for nested classes #1780 #1913 @bkojusner @mike-hunhoff

Breaking Changes

  • remove the SCOPE_* constants in favor of the Scope enum #1764 @williballenthin
  • protobuf: deprecate RuleMetadata.scope in favor of RuleMetadata.scopes @williballenthin
  • protobuf: deprecate Metadata.analysis in favor of Metadata.analysis2 that is dynamic analysis aware @williballenthin
  • update freeze format to v3, adding support for dynamic analysis @williballenthin
  • extractor: ignore DLL name for api features #1815 @mr-tz
  • main: introduce wrapping routines within main for working with CLI args #1813 @williballenthin
  • move functions from capa.main to new capa.loader namespace #1821 @williballenthin
  • proto: add package declaration #1960 @larchchen

New Rules (41)

Bug Fixes

  • ghidra: fix ints_to_bytes performance #1761 @mike-hunhoff
  • binja: improve function call site detection @xusheng6
  • binja: use binaryninja.load to open files @xusheng6
  • binja: bump binja version to 3.5 #1789 @xusheng6
  • elf: better detect ELF OS via GCC .ident directives #1928 @williballenthin
  • elf: better detect ELF OS via Android dependencies #1947 @williballenthin
  • fix setuptools package discovery #1886 @gmacon @mr-tz
  • remove unnecessary scripts/vivisect-py2-vs-py3.sh file #1949 @JCoonradt

capa explorer IDA Pro plugin

  • various integration updates and minor bug fixes

Development

  • update ATT&CK/MBC data for linting #1932 @mr-tz

Developer Notes

With this new release, many classes and concepts have been split up into static (mostly identical to the prior implementations) and dynamic ones. For example, the legacy FeatureExtractor class has been renamed to StaticFeatureExtractor and the DynamicFeatureExtractor has been added.

Starting from version 7.0, we have moved the component responsible for feature extractor from main to a new capabilities' module. Now, users wishing to utilize capa’s feature extraction abilities should use that module instead of importing the relevant logic from the main file.

For sandbox-based feature extractors, we are using Pydantic models. Contributions of more models for other sandboxes are very welcome!

With this release we've reorganized the logic found in main() to localize logic and ease readability and ease changes and integrations. The new "main routines" are expected to be used only within main functions, either capa main or related scripts. These functions should not be invoked from library code.

Beyond copying code around, we've refined the handling of the input file/format/backend. The logic for picking the format and backend is more consistent. We've documented that the input file is not necessarily the sample itself (cape/freeze/etc.) inputs are not actually the sample.

Raw diffs

v7.0.0-beta

2 months ago

This is the beta release of capa v7.0 which was mainly worked on during the Google Summer of Code (GSoC) 2023. A huge shoutout to @colton-gabertan and @yelhamer for their amazing work.

Also a big thanks to the other contributors: @aaronatp, @Aayush-Goel-04, @bkojusner, @doomedraven, @ruppde, and @xusheng6.

New Features

  • add Ghidra backend #1770 #1767 @colton-gabertan @mike-hunhoff
  • add dynamic analysis via CAPE sandbox reports #48 #1535 @yelhamer
    • add call scope #771 @yelhamer
    • add thread scope #1517 @yelhamer
    • add process scope #1517 @yelhamer
    • rules: change meta.scope to meta.scopes @yelhamer
    • protobuf: add Metadata.flavor @williballenthin
  • binja: add support for forwarded exports #1646 @xusheng6
  • binja: add support for symtab names #1504 @xusheng6
  • add com class/interface features #322 @Aayush-goel-04
  • dotnet: emit enclosing class information for nested classes #1780 #1913 @bkojusner @mike-hunhoff

Breaking Changes

  • remove the SCOPE_* constants in favor of the Scope enum #1764 @williballenthin
  • protobuf: deprecate RuleMetadata.scope in favor of RuleMetadata.scopes @williballenthin
  • protobuf: deprecate Metadata.analysis in favor of Metadata.analysis2 that is dynamic analysis aware @williballenthin
  • update freeze format to v3, adding support for dynamic analysis @williballenthin
  • extractor: ignore DLL name for api features #1815 @mr-tz

New Rules (41)

Bug Fixes

  • ghidra: fix ints_to_bytes performance #1761 @mike-hunhoff
  • binja: improve function call site detection @xusheng6
  • binja: use binaryninja.load to open files @xusheng6
  • binja: bump binja version to 3.5 #1789 @xusheng6
  • elf: better detect ELF OS via GCC .ident directives #1928 @williballenthin
  • fix setuptools package discovery #1886 @gmacon @mr-tz

Development

  • update ATT&CK/MBC data for linting #1932 @mr-tz

Developer Notes

With this new release, many classes and concepts have been split up into static (mostly identical to the prior implementations) and dynamic ones. For example, the legacy FeatureExtractor class has been renamed to StaticFeatureExtractor and the DynamicFeatureExtractor has been added.

Starting from version 7.0, we have moved the component responsible for feature extractor from main to a new capabilities' module. Now, users wishing to utilize capa’s feature extraction abilities should use that module instead of importing the relevant logic from the main file.

For sandbox-based feature extractors, we are using Pydantic models. Contributions of more models for other sandboxes are very welcome!

Raw diffs

v6.1.0

7 months ago

capa v6.1.0 is a bug fix release, most notably fixing unhandled exceptions in the capa explorer IDA Pro plugin. @Aayush-Goel-04 put a lot of effort into improving code quality and adding a script for rule authors. The script shows which features are present in a sample but not referenced by any existing rule. You could use this script to find opportunities for new rules.

Speaking of new rules, we have eight additions, coming from Ronnie, Jakub, Moritz, Ervin, and [email protected]!

New Features

  • ELF: implement import and export name extractor #1607 #1608 @Aayush-Goel-04
  • bump pydantic from 1.10.9 to 2.1.1 #1582 @Aayush-Goel-04
  • develop script to highlight features not used during matching #331 @Aayush-Goel-04

New Rules (8)

Modified rules (9)

Renamed rules (1)

Bug Fixes

  • rules: fix forwarded export characteristic #1656 @RonnieSalomonsen
  • Binary Ninja: Fix stack string detection #1473 @xusheng6
  • linter: skip native API check for NtProtectVirtualMemory #1675 @williballenthin
  • OS: detect Android ELF files #1705 @williballenthin
  • ELF: fix parsing of symtab #1704 @williballenthin
  • result document: don't use deprecated pydantic functions #1718 @williballenthin
  • pytest: don't mark IDA tests as pytest tests #1719 @williballenthin

capa explorer IDA Pro plugin

  • fix unhandled exception when resolving rule path #1693 @mike-hunhoff

EDIT: a standalone binary created using Python 3.11 is now available.

Raw diffs

v6.0.0

9 months ago

v6.0.0

capa v6.0 brings many bug fixes and quality improvements, including 64 rule updates and 26 new rules. We're now publishing to PyPI via Trusted Publishing and have migrated to using a pyproject.toml file. @Aayush-Goel-04 contributed a lot of new code across many files, so please welcome them to the project, along with @anders-v @crowface28 @dkelly2e @RonnieSalomonsen and @ejfocampo as first-time rule contributors!

For those that use capa as a library, we've introduced some limited breaking changes that better represent data types (versus less-structured data like dictionaries and strings). With the recent deprecation, we've also dropped support for Python 3.7.

New Features

  • add script to detect feature overlap between new and existing capa rules #1451 @Aayush-Goel-04
  • extract forwarded exports from PE files #1624 @williballenthin
  • extract function and API names from ELF symtab entries @yelhamer https://github.com/mandiant/capa-rules/issues/736
  • use fancy box drawing characters for default output #1586 @williballenthin

Breaking Changes

  • use a class to represent Metadata (not dict) #1411 @Aayush-Goel-04 @manasghandat
  • use pathlib.Path to represent file paths #1534 @Aayush-Goel-04
  • Python 3.8 is now the minimum supported Python version #1578 @williballenthin
  • Require a Contributor License Agreement (CLA) for PRs going forward #1642 @williballenthin

New Rules (26)

Bug Fixes

  • extractor: add a Binary Ninja test that asserts its version #1487 @xusheng6
  • extractor: update Binary Ninja stack string detection after the new constant outlining feature #1473 @xusheng6
  • extractor: update vivisect Arch extraction #1334 @mr-tz
  • extractor: avoid Binary Ninja exception when analyzing certain files #1441 @xusheng6
  • symtab: fix struct.unpack() format for 64-bit ELF files @yelhamer
  • symtab: safeguard against ZeroDivisionError for files containing a symtab with a null entry size @yelhamer
  • improve ELF strtab and needed parsing @mr-tz
  • better handle exceptional cases when parsing ELF files #1458 @Aayush-Goel-04
  • improved testing coverage for Binary Ninja backend #1446 @Aayush-Goel-04
  • add logging and print redirect to tqdm for capa main #749 @Aayush-Goel-04
  • extractor: fix binja installation path detection does not work with Python 3.11
  • tests: refine the IDA test runner script #1513 @williballenthin
  • output: don't leave behind traces of progress bar @williballenthin
  • import-to-ida: fix bug introduced with JSON report changes in v5 #1584 @williballenthin
  • main: don't show spinner when emitting debug messages #1636 @williballenthin

capa explorer IDA Pro plugin

Development

  • update ATT&CK/MBC data for linting #1568 @mr-tz
  • log time taken to analyze each function #1290 @williballenthin
  • tests: make fixture available via conftest.py #1592 @williballenthin
  • publish via PyPI trusted publishing #1491 @williballenthin
  • migrate to pyproject.toml #1301 @williballenthin
  • use pre-commit to invoke linters #1579 @williballenthin

Raw diffs

v5.1.0

1 year ago

capa version 5.1.0 adds a Protocol Buffers (protobuf) format for result documents. Additionally, the Vector35 team contributed a new feature extractor using Binary Ninja. Other new features are a new CLI flag to override the detected operating system, functionality to read and render existing result documents, and a output color format that's easier to read.

Over 25 capa rules have been added and improved.

Thanks for all the support, especially to @xusheng6, @captainGeech42, @ggold7046, @manasghandat, @ooprathamm, @linpeiyu164, @yelhamer, @HongThatCong, @naikordian, @stevemk14ebr, @emtuls, @raymondlleong, @bkojusner, @joren485, and everyone else who submitted bugs and provided feedback!

New Features

  • add protobuf format for result documents #1219 @williballenthin @mr-tz
  • extractor: add Binary Ninja feature extractor @xusheng6
  • new cli flag --os to override auto-detected operating system for a sample @captainGeech42
  • change colour/highlight to "cyan" instead of "blue" for better readability #1384 @ggold7046
  • add new format to parse output json back to capa #1396 @ooprathamm
  • parse ELF symbols' names to guess OS #1403 @yelhamer

New Rules (26)

Bug Fixes

  • extractor: removed '.dynsym' as the library name for ELF imports #1318 @stevemk14ebr
  • extractor: fix vivisect loop detection corner case #1310 @mr-tz
  • match: extend OS characteristic to match OS_ANY to all supported OSes #1324 @mike-hunhoff
  • extractor: fix IDA and vivisect string and bytes features overlap and tests #1327 #1336 @xusheng6

capa explorer IDA Pro plugin

  • fix exception when plugin loaded in IDA hosted under idat #1341 @mike-hunhoff
  • improve embedded PE detection performance and reduce FP potential #1344 @mike-hunhoff

Raw diffs

v5.0.0

1 year ago

This capa version comes with major improvements and additions to better handle .NET binaries. To showcase this we've updated and added over 30 .NET rules.

Additionally, capa now caches its rule set for better performance. The capa explorer also caches its analysis results, so that multiple IDA Pro or plugin invocations don't need to repeat the same analysis.

We have removed the SMDA backend and changed the program return codes to be positive numbers.

Other improvements to highlight include better ELF OS detection, various rendering bug fixes, and enhancements to the feature extraction. We've also added support for Python 3.11.

Thanks for all the support, especially to @jsoref, @bkojusner, @edeca, @richardweiss80, @joren485, @ryantxu1, @mwilliams31, @anushkavirgaonkar, @MalwareMechanic, @Still34, @dzbeck, @johnk3r, and everyone else who submitted bugs and provided feedback!

New Features

  • verify rule metadata format on load #1160 @mr-tz
  • dotnet: emit property features #1168 @anushkavirgaonkar
  • dotnet: emit API features for objects created via the newobj instruction #1186 @mike-hunhoff
  • dotnet: emit API features for generic methods #1231 @mike-hunhoff
  • Python 3.11 support #1192 @williballenthin
  • dotnet: emit calls to/from MethodDef methods #1236 @mike-hunhoff
  • dotnet: emit namespace/class features for ldvirtftn/ldftn instructions #1241 @mike-hunhoff
  • dotnet: emit namespace/class features for type references #1242 @mike-hunhoff
  • dotnet: extract dotnet and pe format #1187 @mr-tz
  • don't render all library rule matches in vverbose output #1174 @mr-tz
  • cache the rule set across invocations for better performance #1212 @williballenthin
  • update ATT&CK/MBC data for linting #1297 @mr-tz

Breaking Changes

  • remove SMDA backend #1062 @williballenthin
  • error return codes are now positive numbers #1269 @mr-tz

New Rules (77)

Bug Fixes

  • render: convert feature attributes to aliased dictionary for vverbose #1152 @mike-hunhoff
  • decouple Token dependency / extractor and features #1139 @mr-tz
  • update pydantic model to guarantee type coercion #1176 @mike-hunhoff
  • do not overwrite version in version.py during PyInstaller build #1169 @mr-tz
  • render: fix vverbose rendering of offsets #1215 @williballenthin
  • elf: better detect OS via GLIBC ABI version needed and dependencies #1221 @williballenthin
  • dotnet: address unhandled exceptions with improved type checking #1230 @mike-hunhoff
  • fix import-to-ida script formatting #1208 @williballenthin
  • render: fix verbose rendering of scopes #1263 @williballenthin
  • rules: better detect invalid rules #1282 @williballenthin
  • show-features: better render strings with embedded whitespace #1267 @williballenthin
  • handle vivisect bug around strings at instruction level, use min length 4 #1271 @williballenthin @mr-tz
  • extractor: guard against invalid "calls from" features #1177 @mr-tz
  • extractor: add format to global features #1258 @mr-tz
  • extractor: discover all strings with length >= 4 #1280 @mr-tz
  • extractor: don't extract byte features for strings #1293 @mr-tz

capa explorer IDA Pro plugin

  • fix: display instruction items #1154 @mr-tz
  • fix: accept only plaintext pasted content #1194 @williballenthin
  • fix: UnboundLocalError #1217 @williballenthin
  • extractor: add support for COFF files and extern functions #1223 @mike-hunhoff
  • doc: improve error messaging and documentation related to capa rule set #1249 @mike-hunhoff
  • fix: assume 32-bit displacement for offsets #1250 @mike-hunhoff
  • generator: refactor caching and matching #1251 @mike-hunhoff
  • fix: improve exception handling to prevent IDA from locking up when errors occur #1262 @mike-hunhoff
  • verify rule metadata using Pydantic #1167 @mr-tz
  • extractor: make read consistent with file object behavior #1254 @mr-tz
  • fix: UnboundLocalError x2 #1302 @mike-hunhoff
  • cache capa results across IDA sessions #1279 @mr-tz

Raw diffs

v4.0.1

1 year ago

Some rules contained invalid metadata fields that caused an error when rendering rule hits. We've updated all rules and enhanced the rule linter to catch such issues.

New Rules (1)

Bug Fixes

  • linter: use pydantic to validate rule metadata #1141 @mike-hunhoff
  • build binaries using PyInstaller no longer overwrites functions in version.py #1136 @mr-tz

Raw diffs

v4.0.0

1 year ago

Version 4 adds support for analyzing .NET executables. capa will autodetect .NET modules, or you can explicitly invoke the new feature extractor via --format dotnet. We've also extended the rule syntax for .NET features including namespace and class.

Additionally, new instruction scope and operand features enable users to create more explicit rules. These features are not backwards compatible. We removed the previously used /x32 and /x64 flavors of number and operand features.

We updated 49 existing rules and added 22 new rules leveraging these new features and characteristics to detect capabilities seen in .NET malware.

More breaking changes include updates to the JSON results document, freeze file format schema (now format version v2), and the internal handling of addresses.

Thanks for all the support, especially to @htnhan, @jtothej, @sara-rn, @anushkavirgaonkar, and @_re_fox!

Deprecation warning: v4.0 will be the last capa version to support the SMDA backend.

New Features

  • add new scope "instruction" for matching mnemonics and operands #767 @williballenthin
  • add new feature "operand[{0, 1, 2}].number" for matching instruction operand immediate values #767 @williballenthin
  • add new feature "operand[{0, 1, 2}].offset" for matching instruction operand offsets #767 @williballenthin
  • extract additional offset/number features in certain circumstances #320 @williballenthin
  • add detection and basic feature extraction for dotnet #987 @mr-tz, @mike-hunhoff, @williballenthin
  • add file string extraction for dotnet files #1012 @mike-hunhoff
  • add file function-name extraction for dotnet files #1015 @mike-hunhoff
  • add unmanaged call characteristic for dotnet files #1023 @mike-hunhoff
  • add mixed mode characteristic feature extraction for dotnet files #1024 @mike-hunhoff
  • emit class and namespace features for dotnet files #1030 @mike-hunhoff
  • render: support Addresses that aren't simple integers, like .NET token+offset #981 @williballenthin
  • document rule tags and branches #1006 @williballenthin, @mr-tz

Breaking Changes

  • instruction scope and operand feature are new and are not backwards compatible with older versions of capa
  • Python 3.7 is now the minimum supported Python version #866 @williballenthin
  • remove /x32 and /x64 flavors of number and operand features #932 @williballenthin
  • the tool now accepts multiple paths to rules, and JSON doc updated accordingly @williballenthin
  • extractors must use handles to identify functions/basic blocks/instructions #981 @williballenthin
  • the freeze file format schema was updated, including format version bump to v2 #986 @williballenthin

Deprecation notice: as described in #937, we plan to remove the SMDA backend for v5. If you rely on this backend, please reach out so we can discuss extending the support for SMDA or transitioning your workflow to use vivisect.

New Rules (30)

Bug Fixes

  • improve handling _ prefix compile/link artifact #924 @mike-hunhoff
  • better detect OS in ELF samples #988 @williballenthin
  • display number feature zero in vverbose #1097 @mike-hunhoff

capa explorer IDA Pro plugin

  • improve file format extraction #918 @mike-hunhoff
  • remove decorators added by IDA to ELF imports #919 @mike-hunhoff
  • bug fixes for Address abstraction #1091 @mike-hunhoff

Raw diffs

v3.2.1

1 year ago

This release bumps the SMDA dependency version to enable installation on Python 3.10.

Bug Fixes

  • update SMDA dependency @mike-hunhoff #922

Raw diffs