Model Estimation by Example

This is the repo for the document found here, based on the raw code repo- Miscellaneous R Code. The document code has been tidied and may have other improvements relative to the original repo, but remains a conceptual/tutorial endeavor to various modeling techniques and algorithms. It provides 'by-hand' demonstrations of various models and algorithms. The goal is to take away some of the mystery of them by providing clean code examples that are easy to run and compare with other tools. The primary functions are usually just a few lines of code, and in almost all cases, comparisons are made to an R package result. A few Python and other language demos as well.

At present the topics covered are:

* Python ** Stan

Models

• Linear Regression *
• Logistic Regression *
• One-factor Mixed Model
• Two-factor Mixed Model
• Mixed Model via ML
• Probit & Bivariate Probit
• Heckman Selection
• Marginal Structural Model
• Tobit Regression
• Cox Survival
• Hurdle Model
• Zero-Inflated Model
• Naive Bayes
• Multinomial Regression
• Ordinal Regression
• Markov Model
• Hidden Markov Model *
• Quantile Regression *
• Cubic Spline Model
• Gaussian Processes
• Neural Network *
• Extreme Learning Machine
• Reproducing Kernel Hilbert Space Regression
• Confirmatory Factor Analysis

Bayesian

• Introduction to Bayesian Methods
• Basics
• Bayesian t-test **
• Bayesian Linear Regression **
• Bayesian Beta Regression **
• Bayesian Mixed Model **
• Bayesian Multilevel Mediation **
• Bayesian IRT **
• Bayesian CFA **
• Bayesian Nonparametric Models **
• Bayesian Stochastic Volatility Model **
• Bayesian Multinomial Models **
• Variational Bayes Regression
• Topic Model

Estimation

• Maximum Likelihood
• Penalized Maximum Likelihood
• L1 (lasso) regularization
• L2 (ridge) regularization
• Newton and IRLS
• Nelder-Mead *
• Expectation-Maximization
• Gradient Descent
• Stochastic Gradient Descent *
• Metropolis Hastings
• Hamiltonian Monte Carlo

* Python ** Stan