Binary Analysis Platform
#887 Llvm 7.0 compatibility #872 Constant tracking analysis #871 Information sensitivity analysis #868 IDA Pro brancher. #866 Bap-elf compilation for ocaml >= 4.05.0 #857 IR Optimization pass #855 Improves reconstructor, symtab, and brancher performance #840 Traps for memory and division operations in Primus #833 Support for x86 CDQ, CDQE, CQO, CWD, CWDE, CBW opcodes #830 BIL passes #821 Interface for binable interval trees #815 Recipes enhancements #813 Mangling of duplicated subroutines #799 Macho loader enhancements #798 Primus Lisp enhancements #795 Llvm 6.0 compatibility
#863 Fixes calls search in reconstructor #856 Fixes the bug that was producing unreachable blocks #853 Cleans /tmp/ida every time ida is called #847 Fixes type inference #841 Several Primus fixes #836 Fixes packed x86 instructions #832 Fixes function names in docstrings #827 Fixes compilations issue on 32 bit machine #817 Hardens IDA plugin #804 Removes duplicated sections in disassemble output #796 Fixes paths comprehension in api plugin #792 Restores the section view #791 Produces correct callgraphs if there are no calls in a program #789 Fixes BIL normalization procedure #733 Fixed bugs in x86 shift instructions
Graphlib library
Graphlib is a generic library that extends a well known
OCamlGraph library. Graphlib uses its own, more reach and modern,
Graph interface that is isomorphic to OCamlGraph's Sigs.P
signature for persistant graphs. This interface is developed
according to the Janestreet's style guidlines and depends on
Core_kernel library. Other than the new interface, Graphlib
provides several graph implementations, and generic algorithms. To
make our algorithms polymorphic over chosen graph representation we
use first-class modules, instead of functors, that makes library
syntactically more light weight.
Refined IR.
phi and arg terms were refined. a phi term now is a discriminated set of expressions, and arguments are made more like a defintions.
SSA form
A transformation to a semipruned SSA form was added to a library
Changed Var interface.
Variables can now be versioned, that plays well with SSA form. A version is used to represent the same variable but at different point of time.
Extended and fixed Trie data structure
Several bug-fixes to a Trie data structure were made. Added
walk function, that allows to perform generic queries over a trie.
Also added prefix and suffix variants of String tries, as well as
provided a functor to create tries with an array keys.
Redesigned abi interface.
Now abi works on an IR level.
Added argument reconstruction algorithms.
Two argument reconstructions algorithm, based on a new abi
interface, were added - for ARM gnueabi ABI and System V amd64 ABI.
Currently only parameters passed via registers are reconstructed.
Added free_vars algorithm to BIL and BIR terms.
BAP IR is introduced
BAP Intermediate Representation is based on BAP Instruction Language and is a semigraphical representation of a program.
See documentation and following PR's for more information. a2a4621df7c5b25d85c04665732423992e8def98 74cdee48818225e8b43d39803c97471903ef6d1f
Refactored structure of the Project
Module Project now a proper entry point to the library.
Many stuff from bap utility moved there.
See 96bd334a0d8af17a6dfd21eff9ec710d448f13e8 for more details.
This is a breaking change. It hides project record and removes
access to some information, that was previously marked as deprecated:
Instead of old symbols table we now have a better interface, see below. Instead of base, we now represent all memory as an interval map (Memmap).
New model for symbols
Previosly symbols were modeled as contiguous chunk of memory,
marked with name. Moreover, data sharing between different symbols
weren't allowed. Since this release, symbols can be a noncontiguous,
and share data. A new interface is implemented in Symtab module.
Plugins dependency and autoloading
Plugins now can now specify dependencies to other plugins, that may be auto-loaded by the library. See db2a175ba8e6708753a06a2428940c857a1910ec
Extended BIL helpers See 65f472c08d27020a6570b7992b93397346251d1e
Exposed ELF library
Fixed segment/section/region name hell See 9a574498392c6a13606c9d202037daf137bb780c
New universal values library
The library is based on Core_kernel's Univ, but with addition of serialization, comparison and pretty-printing. See 383003d60baa3434dd4cd8c894e1d8c2e889b4a2
Added bap-fsi-benchmark utility
80382114f395bcf45925ae2e4bc5b9aac5bba4e7
Fixed BIL piqi serialization
2a5c4671468c5a2699b6007a8af3fda8867e8eb8
Fixed installation on more recent ubuntu
By defaulting LLVM version to 3.4 (and more clever searching procedure)
Lot's of bugfixes and small extensions
BAP structure is refactored
Complex hierarchy is now hidden under one umbrella bap.mli.
Bap_* modules are marked as internal and is no more installable
and, thus, they do not polute the namespace. This will of course,
break the code that used this internal modules. It is intended
behavior.
New documentation generator
camlp4 messes with mli files, rendering autogenerated API
unreadable. To mitigate this, a small bapdoc tool was written
that preprocess file and removes all syntax extensions, and then
run ocamldoc on a preprocessed file. The tool was integrated
with build system.
Disabled peer checking when downloading signatures
Fixed assembly pretty-printing
Fixed reading PE-64
Fixed Block.dfs exhaustiveness issue.
Also Block.dfs now has an order parameter, allowing to
choose between post-order and pre-order traversal.
Fixed to_graph issue, i.e., #181
Fixed bapbuild double linking, see #193
Ordered block destinations, see #191
Fixed an issue in x86 lifter, see #198
Fixed interaction with IDA, see #189
New loader backed with LLVM BAP now have another loader (image reader), that supports MACH-O, ELF, COFF, PE. This loader is backed with LLVM library.
Online plugin system
New extension point is added - "bap.project". Plugins marked with
this plugin system will not be loaded automatically when
Plugins.load is called, instead, they can be loaded dynamically
(or online, hence the title), by using -l option to the bap
utility. After being loaded the plugin is applied to a project
data structure that contains all information about disassembled
binary. Plugin can functionally update this data structure, to
push information to other plugins or back to the bap utility.
In addition to a common way of creating plugins with oasis, we
extended bapbuild utility with a new rule the will product a
plugin file. This is just a shared library underneath the hood,
and you can load a plugin, created with this method directly,
without installing it anywhere. bap utility will try to find the
plugin, specified with -l option in a current folder, then in all
folders specified in BAP_PLUGIN_PATH environment variable, and,
finally in the system, using ocamlfind.
In order to provide a typesafe way of interacting between plugins, we added extensible variants to BAP. But instead of using one from the 4.02, we're using universal types, based on that one, that Core library provides. First of all this is more portable, second it is more explicit and a little bit more safe.
New ABI and CPU interfaces
Modules that implements CPU interface are used to describe
particular CPU in BIL terminology, e.g., it tells which variable
corresponds to which register, flag, etc. To obtain such module,
one should use target_of_cpu function.
ABI is used to capture the procedure abstraction, starting from calling convetions and stack frame structure and ending with special function handling and support for different data-types.
See d5cab1a5e122719b4a3b1ece2b1bc44f3f93095a for more information and examples.
Bap-objdump renamed to bap
bap-objdump has outgrown its name. Actually it was never really a
bap-objdump at all. From now, it is just an entry point to the bap as
platform. We will later unite bap with other utilities, to make them
subcommands, e.g. bap byteweight.
Cleanup of BIL modules
Now there is a separation between BIL fur uns, and BIL fur
OCaml. For writing BIL programs (as EDSL in OCaml) one should use
Bil module, e.g. Bil.(x = y) will evaluate to a BIL
expression. For using BIL entities as OCaml values, one should use
corresponding module, e.g. Exp.(x = y) will compare to expressions
and evaluate to a value of type bool.
Enhanced IDA integration
IDA intergation is now more robust. We switched to IDA-32 by default,
since 64-bit version doesn't support decompiler. Also bap utility
can now output IDA python scripts. And bap plugins can annotate project
with python commands, that later will be dumped into the script.
In ARM switched to ARMv7 by default
Introduce LNF algorithm and Sema library A new layer of BAP is started in this release. This would be a third pass of decompilation, where the semantic model of program will be built. Currently, there is nothing really interesting here, e.g., an implementation of the Loop nesting forest, that is not very usable right now. But the next release, will be dedicated to this layer. So, stay tuned.
Add support for OCamlGraph Now we provide a helper utilities for those who would like to use ocamlgraph library for analysis.
Extended bap-mc utility
bap-mc utility now prints results in plethora of formats,
including protocol buffers, from the piqi library, that was revived
by Kenneth Miller.
Interval trees, aka memory maps For working with arbitrary overlapping memory regions we now have a memory map data structure, aka interval trees, segment trees, etc. It is based on AVL trees, and performs logarithmic searches.
Simplified CI We put Travis on a diet. Now only 4 machines with 20 ETA for all test suites to pass. (Instead of 8 * 40).