FastAD Save
FastAD is a C++ implementation of automatic differentiation both forward and reverse mode.
FastAD
Table of contents
- FastAD
Overview
FastAD is a header-only C++ template library for automatic differentiation supporting both forward and reverse mode. It utilizes the latest features in C++17 and expression templates for efficient computation. FastAD is unique for the following:
Intuitive syntax
Syntax choice is very important for C++ developers. Our philosophy is that syntax should be as similar as possible to mathematical notation. This makes FastAD easy to use and allow users to write readable, intuitive, and simple code. See User Guide for more details.
Robustness
FastAD has been heavily unit-tested with high test coverage followed by a few integration tests. A variety of functions have been tested against analytical solutions; at machine-level precision, the derivatives coincide.
Memory Efficiency
FastAD is written to be incredibly efficient with memory usage and cache hits. The main overhead of most AD libraries is the tape, which stores adjoints. Using expression template techniques, and smarter memory management, we can significantly reduce this overhead.
Speed
Speed is the utmost critical aspect of any AD library. FastAD has been proven to be extremely fast, which inspired the name of this library. Benchmark shows over orders of magnitude improvement from existing libraries such as Adept, Stan Math Library, ADOL-C, CppAD, and Sacado (see our separate benchmark repositor ADBenchmark). Moreover, it also shows 10x improvement from the naive (and often inaccurate) finite-difference method.
Installation
First, clone the repo:
git clone https://github.com/JamesYang007/FastAD.git ~/FastAD
From here on, we will refer to the cloned directory as workspace_dir
(in the example above, workspace_dir is ~/FastAD).
The library has the following dependencies:
- Eigen3.3
- GoogleTest (dev only)
- Google Benchmark (dev only)
General Users
If the user already has Eigen3.3 installed in their system, they can omit the following step. For general users, if they wish to install Eigen locally, they can run
./setup.sh
from workspace_dir.
This will install Eigen3.3 into workspace_dir/libs/eigen-3.3.7/build.
For those who want to install FastAD globally into the system, simply run:
./install.sh
This will build and install the header files into the system.
For users who want to install FastAD locally, run the following from workspace_dir:
mkdir -p build && cd build
cmake -DCMAKE_INSTALL_PREFIX=. ..
make install
One can set the CMAKE_INSTALL_PREFIX to anything.
This example will install the library in workspace_dir/build.
For users that want to integrate FastAD in their own CMakeLists, use FetchContent (CMake >= 3.11)
include(FetchContent)
FetchContent_Declare(
FastAD
GIT_REPOSITORY https://github.com/JamesYang007/FastAD
GIT_TAG v3.2.1
GIT_SHALLOW TRUE
GIT_PROGRESS TRUE)
FetchContent_MakeAvailable(FastAD)
# Further link target 'FastAD'
Developers
Run the following to install all of the dependencies locally:
./setup.sh dev
To build the library, run the following:
./clean-build.sh <debug/release> [other CMake flags...]
Here are the following options one can specify as a CMake flag -D...=ON
(replace ... with any of the following):
- FASTAD_ENABLE_TEST (builds tests)
- FASTAD_ENABLE_BENCHMARK (builds benchmarks)
- FASTAD_ENABLE_EXAMPLE (builds examples)
By default, the flags are OFF.
Note that this only builds and does not install the library.
To run tests, execute the following:
cd build/<debug/release>
ctest -j6
To run benchmarks, change directory to
build/<debug/release>/benchmark and run any one of the executables.
Integration
CMake
If your project is built using CMake, add the following to CMakeLists.txt in the root directory:
find_package(FastAD CONFIG REQUIRED)
If you installed the library locally, say path_to_install, then add the following:
find_package(FastAD CONFIG REQUIRED HINTS path_to_install/share)
For any program that requires FastAD,
use target_link_libraries to link with FastAD::FastAD.
An example project that uses FastAD as a dependency may have a CMakeLists.txt that looks like this:
project("MyProject")
find_package(FastAD CONFIG REQUIRED)
add_executable(main src/main.cpp)
target_link_libraries(main FastAD::FastAD)
Others
Simply add the following flag when compiling your program:
-Ipath_to_install/include
An example build command would be:
g++ main.cpp -Ipath_to_install/include
If Eigen3.3 was installed locally, you must provide its path as well.
User Guide
The only header the user needs to include is fastad.
Forward Mode
Forward mode is extremely simple to use.
The only class a user will need to deal with is ForwardVar<T>,
where T is the underlying data type (usually double).
The API only exposes getters and setters:
ForwardVar<double> v; // initialize value and adjoint to 0
v.set_value(1.); // value is now 1.
double r = v.get_value(); // r is now 1.
v.set_adjoint(1.); // adjoint is now 1.
double s = v.get_adjoint(); // s is now 1.
The rest of the work has already been done by the library with operator overloading.
Here is an example program that differentiates a complicated function:
#include <fastad>
#include <iostream>
int main()
{
using namespace ad;
ForwardVar<double> w1(0.), w2(1.);
w1.set_adjoint(1.); // differentiate w.r.t. w1
ForwardVar<double> w3 = w1 * sin(w2);
ForwardVar<double> w4 = w3 + w1 * w2;
ForwardVar<double> w5 = exp(w4 * w3);
std::cout << "f(x, y) = exp((x * sin(y) + x * y) * x * sin(y))\n"
<< "df/dx = " << w5.get_adjoint() << std::endl;
return 0;
}
We initialize w1 and w2 with values 0. and 1., respectively.
We set the adjoint of w1 to 1. to indicate that we are differentiating w.r.t. w1.
By default, all adjoints of ForwardVar are set to 0..
This indicates that we will be differentiating in the direction of (1.,0.), i.e. partial derivative w.r.t. w1.
Note that user could also set the adjoint for w2,
but this will compute the directional derivative multiplied by the norm of (w1, w2).
After computing the desired expression, we get the directional derivative
by calling get_adjoint() on the final ForwardVar object.
Reverse Mode
Basic Usage
The most basic usage simply requires users to create Var<T, ShapeType> objects.
T denotes the underlying value type (usually double).
ShapeType denotes the general shape of the variable.
It must be one of ad::scl, ad::vec, ad::mat corresponding to
scalar, (column) vector, and matrix, respectively.
Var<double, scl> x;
Var<double, vec> v(5); // set size to 5
Var<double, mat> m(2, 3); // set shape to 2x3
From here, one can create complicated expressions by invoking a wide range of functions (see Quick Reference for a full list of expression builders).
As an example, here is an expression to differentiate sin(x) + cos(v):
auto expr = (sin(x) + cos(v));
Note that this represents a vector expression, since sin(x) is a scalar expression
but cos(v) is a vectorized function on a vector, which is again a vector expression.
Before we differentiate, the expression is required to "bind" to a storage for the values and adjoints of intermediate expression nodes. The reason for this design is for speed purposes and cache hits. If the user wishes to manage this storage, they can do this:
auto size_pack = expr.bind_cache_size();
std::vector<double> val_buf(size_pack(0));
std::vector<double> adj_buf(size_pack(1));
expr.bind_cache({val_buf.data(), adj_buf.data()});
The bind_cache_size() will return exactly how many doubles are needed
for values and adjoints, respectively, of type util::SizePack, which is an alias for Eigen::Array<size_t, 2, 1>.
and bind(util::PtrPack<double>) will bind itself to that region of memory.
It is encouraged to create the pointer pack object using initializer list as shown above.
This pattern occurs so often that if the user does not care about managing this,
they should use the following helper function:
auto expr_bound = ad::bind(sin(x) + cos(v));
ad::bind will return a wrapper class that wraps the expression
and at construction binds it to a privately owned storage
in the same way described above.
If the expression is not bound to any storage, it will lead to segfault!
To differentiate the expression, simply call the following:
auto f = ad::autodiff(expr_bound, seed);
// or if the raw expression is manually bound,
auto f = ad::autodiff(expr, seed);
where seed is the initial adjoint for the root of the expression.
If the expression is scalar, seed is a literal (double)
and the default value is 1, so the user does not have to input anything.
If the expression is multi-dimensional,
seed does not have a default value,
must be of type Eigen::Array,
and must have the same dimensions as the expression.
autodiff will return the evaluated function value.
This return value is T if it is a scalar expression, and otherwise,
Eigen::Map<Eigen::Matrix<T, Eigen::Dynamic, ...>> where ... depends on
the shape of the expression (1 if vector, Eigen::Dynamic if matrix).
You can retrieve the adjoints by calling get_adj(i,j) or
get_adj() (with no arguments) from x, v like so:
x.get_adj(0,0); // (1) get adjoint for x
v.get_adj(2,0); // (2) get adjoint for v at index 2
x.get_adj(); // (3) get full adjoint (same as (1))
v.get_adj(); // (4) get full adjoint
The full code for this example is the following:
#include <fastad>
#include <iostream>
int main()
{
using namespace ad;
Var<double, scl> x(2);
Var<double, vec> v(5);
// randomly generate values for v
v.get().setRandom();
// create AD expression bound to storage
auto expr_bound = bind(sin(x) + cos(v));
// seed to get gradient of function at index 2
Eigen::Array<double, Eigen::Dynamic, 1> seed(v.size());
seed.setZero();
seed[2] = 1;
// differentiate
auto f = autodiff(expr_bound, seed);
std::cout << x.get_adj() << std::endl;
std::cout << v.get_adj(2,0) << std::endl;
return 0;
}
Note: once you have differentiated an expression,
you must reset the adjoints of all variables to 0 before differentiating again.
This includes placeholder variables (see below).
To that end, we provide a member function for Var called reset_adj().
Here is a more complicated example:
#include <fastad>
int main()
{
Var<double, vec> v1(6);
Var<double, vec> v2(5);
Var<double, mat> M(5, 6);
Var<double, vec> w(5);
Var<double, scl> r;
auto& v1_raw = v1.get(); // Eigen::Map
auto& v2_raw = v2.get(); // Eigen::Map
auto& M_raw = M.get(); // Eigen::Map
// initialize...
auto expr = bind((
w = ad::dot(M, v1) + v2,
r = sum(w) * sum(w * v2)
));
autodiff(expr);
std::cout << v1.get_adj(0,0) << std::endl; // adjoint of v1 at index 0
std::cout << v2.get_adj(1,0) << std::endl; // adjoint of v2 at index 1
std::cout << M.get_adj(1,2) << std::endl; // adjoint of M at index (1,2)
return 0;
}
Placeholder
In the previous example, we used a placeholder expression,
which is of the form v = expr.
You can use placeholders to greatly speed up the performance
and also save a lot of memory.
Consider the following expression:
auto expr = (sin(x) + cos(v) + sum(cos(v)));
When there are common expressions (like cos(v)),
they will be evaluated multiple times unnecessarily.
Placeholder expressions are created by using operator= with
a Var and an expression:
Var<double, scl> x;
Var<double, vec> v(5);
Var<double, vec> w(v.size());
auto expr = (
w = cos(v),
sin(x) + w + sum(w)
);
This will only evaluate cos(v) once, and reuse the results
for the subsequent expressions by using w.
While this is not specific to placeholder expressions,
operator, is usually invoked to "glue" many placeholder expressions.
However, one can certainly glue any kinds of expressions, if they wish.
Advanced Usage
For advanced users who need to get more low-level control over
the memory for values and adjoints for all variables, they can use
VarView<T, ShapeType>.
All of the discussion above holds for VarView objects.
In fact, when we build an expression out of Var of VarView,
we convert all of them to VarViews so that the expression is solely a viewer.
VarView objects do not own the values and adjoints, but views them.
Here is an example program that binds the viewers to a contiguous chunk of memory:
VarView<double, scl> x;
VarView<double, vec> v(3);
VarView<double, vec> w(3);
std::vector<double> vals(x.size() + v.size());
std::vector<double> adjs(x.size() + v.size());
std::vector<double> w_vals(w.size());
std::vector<double> w_adjs(w.size());
// x binds to the first element of storages
double* val_next = x.bind(vals.data());
double* adj_next = x.bind_adj(adjs.data());
// v binds starting from 2nd element of storages
v.bind(val_next);
v.bind_adj(adj_next);
// bind placeholders to a separate storage region
w.bind(w_vals.data());
w.bind_adj(w_adjs.data());
auto expr = (
w = cos(v),
sin(x) + w + sum(w)
);
Applications
Black-Scholes Put-Call Option Pricing
The following is an example of computing deltas in Black-Scholes model using FastAD. This example was taken from the autodiff library in boost.
#include <fastad>
#include <iostream>
enum class option_type {
call, put
};
// Standard Normal CDF
template <class T>
inline auto Phi(const T& x)
{
return 0.5 * (ad::erf(x / std::sqrt(2.)) + 1.);
}
// Generates expression that computes Black-Scholes option price
template <option_type cp, class Price, class Cache>
auto black_scholes_option_price(const Price& S,
double K,
double sigma,
double tau,
double r,
Cache& cache)
{
cache.resize(3);
double PV = K * std::exp(-r * tau);
auto common_expr = (
cache[0] = ad::log(S / K),
cache[1] = (cache[0] + ((r + sigma * sigma / 2.) * tau)) /
(sigma * std::sqrt(tau)),
cache[2] = cache[1] - (sigma * std::sqrt(tau))
);
if constexpr (cp == option_type::call) {
return (common_expr,
Phi(cache[1]) * S - Phi(cache[2]) * PV);
} else {
return (common_expr,
Phi(-cache[2]) * PV - Phi(-cache[1]) * S);
}
}
int main()
{
double K = 100.0;
double sigma = 5;
double tau = 30.0 / 365;
double r = 1.25 / 100;
ad::Var<double> S(105);
std::vector<ad::Var<double>> cache;
auto call_expr = ad::bind(
black_scholes_option_price<option_type::call>(
S, K, sigma, tau, r, cache));
double call_price = ad::autodiff(call_expr);
std::cout << call_price << std::endl;
std::cout << S.get_adj() << std::endl;
// reset adjoints before differentiating again
S.reset_adj();
for (auto& c : cache) c.reset_adj();
auto put_expr = ad::bind(
black_scholes_option_price<option_type::put>(
S, K, sigma, tau, r, cache));
double put_price = ad::autodiff(put_expr);
std::cout << put_price << std::endl;
std::cout << S.get_adj() << std::endl;
return 0;
}
We observed the same output as the one shown in boost for the prices and deltas (S's adjoints).
Quadratic Expression Differential
In ML applications, user usually provide a full parameter vector to an optimizer and write an objective function to calculate objective value and gradient. The gradient pointer is provided by the optimizer and user fill values. Then it will be more convinent to use VarView to bind the gradient pointer as buffer.
Here is an example of differentiating a quadratic expression x^T*Sigma*x using VarView.
#include <iostream>
#include "fastad"
#include <Eigen/src/Core/Matrix.h>
int main() {
using namespace ad;
// Generating buffer.
Eigen::MatrixXd x_data(2, 1);
x_data << 0.5, 0.6;
Eigen::MatrixXd x_adj(2, 1);
x_adj.setZero(); // Set adjoints to zeros.
// Initialize variable.
VarView<double, mat> x(x_data.data(), x_adj.data(), 2, 1);
// Initialize matrix.
Eigen::MatrixXd _Sigma(2, 2);
_Sigma << 2, 3, 3, 6;
std::cout << _Sigma << std::endl;
auto Sigma = constant(_Sigma);
// Quadratic expression: x^T*Sigma*x
auto expr = bind(dot(dot(transpose(x), Sigma), x));
// Seed
Eigen::MatrixXd seed(1, 1);
seed.setOnes(); // Usually seed is 1. DONT'T FORGET!
// Auto differential.
auto f = autodiff(expr, seed.array());
// Print results.
std::cout << "f: " << f << std::endl;
std::cout << x.get() << std::endl; //[0.5, 0.6]
std::cout << x.get_adj() << std::endl; //[5.6, 10.2]
return 0;
}
Simple Linear Regression Model
In a regression model, one has many rows of data. A loop is needed to calculate loss of each row.
#include "fastad"
#include <iostream>
int main() {
using namespace ad;
// Create data matrix.
Eigen::MatrixXd X(5, 2);
X << 1, 10, 2, 20, 3, 30, 4, 40, 5, 50;
Eigen::VectorXd y(5);
y << 32, 64, 96, 128, 160; // y=2*x1+3*x2
// Generating buffer.
Eigen::MatrixXd theta_data(2, 1);
theta_data << 1, 2;
Eigen::MatrixXd theta_adj(2, 1);
theta_adj.setZero(); // Set adjoints to zeros.
// Initialize variable.
VarView<double, mat> theta(theta_data.data(), theta_adj.data(), 2, 1);
// Create expr. Use a row buffer to store data. Then we only need to manipulate data when
// looping.
Eigen::MatrixXd x_row_buffer = X.row(0);
auto xi = constant_view(x_row_buffer.data(), 1, X.cols());
Eigen::MatrixXd y_row_buffer = y.row(0);
auto yi = constant_view(y_row_buffer.data(), 1, y.cols());
auto expr = bind(pow<2>(yi - dot(xi, theta)));
// Seed
Eigen::MatrixXd seed(1, 1);
seed.setOnes(); // Usually seed is 1. DONT'T FORGET!
// Loop over each row to calulate loss.
double loss = 0;
for (int i = 0; i < X.rows(); ++i) {
x_row_buffer = X.row(i);
y_row_buffer = y.row(i);
auto f = autodiff(expr, seed.array());
loss += f.coeff(0);
}
// Print results.
std::cout << "loss: " << loss << std::endl; // 6655
std::cout << theta.get() << std::endl; //[1, 2]
std::cout << theta.get_adj() << std::endl; //[-1210, -12100]
theta_adj.setZero(); // Reset differential to zero after one full pass.
return 0;
}
3-layer Neural Network with Jump Connection
Here is an example of 3-layer neural network. The result has been confirmed by symbolic differential using Mathematica. See test/reverse/util/GenTestData.nb for Mathematica code used.
There is also a full independent example that using ceres to train this network in examples/ceres_3layer_neural_net. ceres has it's own automatic differential tool for non-linear least square problem which is unavailiable for the more versatile GradientProbelm. That's why we need FastAD. To run this example, you need to install and Eigen and ceres.
#include "fastad"
#include <iostream>
using namespace ad;
// A3*s(A2*s(A1*x+b1)+b2+(A1*x+b1))+b3
struct NN {
Eigen::MatrixXd X, y;
NN(const Eigen::MatrixXd &X, const Eigen::VectorXd &y)
: X(X), y(y){
};
double loss(const double *parm, double *grad) {
Eigen::Map<Eigen::VectorXd> g(grad, 25);
g.setZero();
//Create variables.
VarView<double, mat> A1(const_cast<double *>(parm), grad, 3, 2);
VarView<double, mat> b1(const_cast<double *>(parm + 6), grad + 6, 3, 1);
VarView<double, mat> A2(const_cast<double *>(parm + 9), grad + 9, 3, 3);
VarView<double, mat> b2(const_cast<double *>(parm + 18), grad + 18, 3, 1);
VarView<double, mat> A3(const_cast<double *>(parm + 21), grad + 21, 1, 3);
VarView<double, mat> b3(const_cast<double *>(parm + 24), grad + 24, 1, 1);
// Data buffer
Eigen::VectorXd x_row_buffer = X.row(0);
auto xi = constant_view(x_row_buffer.data(), X.cols(), 1);
Eigen::VectorXd y_row_buffer = y.row(0);
auto yi = constant_view(y_row_buffer.data(), y.cols(), 1);
// Expression
auto x1 = dot(A1, xi) + b1;
auto y1 = sigmoid(x1);
auto y2 = sigmoid(dot(A2, y1) + b2) + x1;
auto y3 = dot(A3, y2) + b3;
auto residual_norm2 = pow<2>(yi - y3);
auto expr = bind(residual_norm2);
// Seed
Eigen::MatrixXd seed(1, 1);
seed.setOnes(); // Usually seed is 1. DONT'T FORGET!
// Loop over each row to calulate loss.
double loss = 0;
for (int i = 0; i < X.rows(); ++i) {
x_row_buffer = X.row(i);
y_row_buffer = y.row(i);
auto f = autodiff(expr, seed.array());
loss += f.coeff(0);
}
return loss;
};
};
int main() {
// Create data matrix.
Eigen::MatrixXd X(5, 2);
X << 1, 10, 2, 20, 3, 30, 4, 40, 5, 50;
Eigen::VectorXd y(5);
y << 32, 64, 96, 128, 160; // y=2*x1+3*x2
// Generating parameter buffer and NN.
Eigen::VectorXd parm_data(25);
parm_data << 0.043984, 0.960126, -0.520941, -0.800526, -0.0287914, 0.635809, 0.584603,
-0.443382, 0.224304, 0.97505, -0.824084, 0.2363, 0.666392, -0.498828, -0.781428, -0.911053,
-0.230156, -0.136367, 0.263425, 0.841535, 0.920342, 0.65629, 0.848248, -0.748697, 0.21522;
Eigen::VectorXd grad_data(25);
grad_data.setZero(); // Set adjoints to zeros.
NN net(X, y);
auto loss = net.loss(parm_data.data(), grad_data.data());
// Print results.
std::cout << "loss: " << loss << std::endl; // 6655
std::cout << "parm: " << parm_data << std::endl; //[1, 2]
std::cout << "diff: " << grad_data << std::endl; //[-1210, -12100]
return 0;
}
Quick Reference
Forward
ForwardVar<T>:
- class representing a variable for forward AD
-
set_value(T x): sets value to x -
get_value(): gets underlying value -
set_adjoint(T x): sets adjoint to x -
get_adjoint(): gets underlying adjoint
Unary Functions:
- unary minus:
operator- - trig functions:
sin, cos, tan, asin, acos, atan - others:
exp, log, sqrt
Operators:
- binary:
+,-,*,/ - increment:
+=
Reverse
Shape Types:
-
ad::scl, ad::vec, ad::mat
VarView<T, ShapeType=scl>:
- This is only useful for users who really want to optimize for performance
-
ShapeTypemust be one of the types listed above -
Tis the underlying value type -
VarView(T* v, T* a, rows=1, cols=1):- constructs to view values starting from v, adjoints starting from a, and has the shape of rows x cols.
- vector shapes must pass rows
- matrix shapes must pass both rows and cols
-
VarView()- constructs with nullptrs
-
.bind(T* begin): views values starting from begin -
.bind_adj(T* begin): views adjoints starting from begin
Var<T, ShapeType=scl>:
- A
Varis aVarView(views itself) - Main difference with
VarViewis that it owns the values and adjoints - Users will primarily use this class to represent AD variables.
- API is same as
VarView
Unary Functions (vectorized if multi-dimensional):
- unary minus:
operator- - trig functions:
sin, cos, tan, asin, acos, atan - Hyperbolic:
sinh, cosh, tanh - Neural network activations:
sigmoid - others:
exp, log, sqrt, erf
Operators:
- binary:
+,-,*,/ - modification:
+=,-=,*=,/= - comparison:
<,<=,>,>=,==,!=,&&,||- Note:
&&and||are undefined behavior for multi-dimensional non-boolean expressions
- Note:
- placeholder:
operator=- only overloaded for
VarViewexpressions
- only overloaded for
- glue:
operator,- any expressions can be "glued" using this operator
- unary minus:
operator- - trig functions:
sin, cos, tan, asin, acos, atan - others:
exp, log, sqrt
Special Expressions:
-
ad::constant(T): -
ad::constant(const Eigen::Vector<T, Eigen::Dynamic, 1>&): -
ad::constant(const Eigen::Matrix<T, Eigen::Dynamic, Eigen::Dynamic>&): -
ad::constant_view(T*): -
ad::constant_view(T*, rows): -
ad::constant_view(T*, rows, cols): -
ad::det<policy>(m):- determinant of matrix
m -
policymust be one of:DetFullPivLU,DetLDLT,DetLLT- see
Eigendocumentation for each of these (without the prefixDet) for when they apply.
- see
- determinant of matrix
-
ad::dot(m, v):- represents matrix product with a matrix and a (column) vector
-
ad::for_each(begin, end, f):- generalization of operator,
- represents evaluating expressions generated by
fwhen fed with elements frombegintoend.
-
ad::if_else(cond, if, else):- represents an if-else statement
-
condMUST be a scalar expression -
ifandelsemust have the exact same shape
-
ad::log_det<policy>(m)- same as
det<policy>(m)but computes log-abs-determinant -
policymust be one of:LogDetFullPivLU,LogDetLDLT,LogDetLLT
- same as
-
ad::norm(v):- represents the squared norm of a vector or Frobenius norm for matrix
-
ad::pow<n>(e):- compile-time known, integer-powered expression
-
ad::prod(begin, end, f):- represents the product of expressions generated by
fwhen fed with elements frombegintoend.
- represents the product of expressions generated by
-
ad::prod(e):- represents the product of all elements of the expression
e - e.g. if
eis a vector expression, it represents the product of all its elements.
- represents the product of all elements of the expression
-
ad::sum(begin, end, f): -
ad::sum(e):- same as prod but represents summation
-
ad::transpose(e):- matrix or vector transpose.
Stats Expressions: All log-pdfs are adjusted to omit constants. Parameters can have various combinations of shapes and follow the usual vectorized notion.
-
ad::bernoulli(x, p) -
ad::cauchy_adj_log_pdf(x, loc, scale) -
ad::normal_adj_log_pdf(x, mu, s) -
ad::uniform_adj_log_pdf(x, min, max) -
ad::wishart_adj_log_pdf(X, V, n)
Contact
If you have any questions about FastAD, please open an issue. When opening an issue, please describe in the fullest detail with a minimal example to recreate the problem.
For other general questions that cannot be resolved through opening issues, feel free to send me an email.
Contributors
| James Yang | Kent Hall | Jean-Christophe Ruel | ZhouYao |
|---|---|---|---|
 |
 |
 |
|
github.com/JamesYang007 |
github.com/kentjhall |
github.com/jeanchristopheruel |
github.com/kilasuelika |
Third Party Tools
Many third party tools were used for this project.
- Clang: main compiler used for development.
- CMake: build automation.
- Codacy: rigorous code analysis.
- Coveralls: for measuring and uploading code coverage.
- Cpp Coveralls: for measuring code coverage in Coveralls.
- Eigen: matrix library that we wrapped to create AD expressions.
- GCC: compiler used to develop in linux environment.
- Github Changelog Generator: generate CHANGELOG.
- Google Benchmark: benchmark against various methods.
- GoogleTest: unit-test and integration-test.
- Travis: continuous integration for Linux and MacOS. See .travis.yml for more details.
- Valgrind: check memory leak/error.
License
- MIT license
- Copyright 2019 ©JamesYang007.
Open Source Agenda Badge
