fuzzball

Scala fuzzer. ~150-160 bugs found in Dotty (some reports have multiple bugs) so far. Powered by a two-layer LSTM (400 units) network trained only for a couple hours.

Installing

pip3 install torch numpy

Running

python3 -u generate.py --temperature 1.0 --count 30 --predict_len 3000 > out.txt
python3 postprocess.py --samples 3 --maxids 8 out.txt results/

Parameters

High temperature (≫ 1.0) - lots of random noise, most samples will be syntactically incorrect. Low temperature (≈ 0.0) - little noise, most samples will be syntactically correct.

Keep predict_len within 300-3000. If you want only very short snippets, set it low (80-150).

count is the number of predict_len-long chunks to generate, controls the amount of data that will be generated.

maxids controls the maximum number of identifiers in snippets.

Each sample as generated by generate.py will further produce samples samples after proprocess.pying.

Examples

Some snippets generated by the fuzzer:

abstract class i0[+I1] {
def +[I2[I2]: i0[_], i3 <: I2[I2]] = (???) :: i3
val I4 = new i6
val i6: i6 = I5
i6
}

object Boolean {}
trait I0[I0]
object I1 {
def I1[I0](I1: => Int, I2: I1 { type I1 <: I0[I2] } with I0.I0[_]#I0[I1] = new I0[I1])#i3#i3 = new I0[I1] {}
}

abstract class I0() {
val I1 = 1 + 1; case (I1: Nil) => 1;
case Some(I1) =>
I1
}.isInstanceOf[Option][String]{ new I0();
val I1 = ;
def I1: Int = 0 { i3 =>
final def i2: Int => Any = I1
}
}

Note that not all snippets will properly parse, and the vast majority of them won't typecheck. It will generate a limited subset of Scala (roughly Scala2 with some Dotty-specific keywords here and there).

Testing on dotty

Add a new test to CompilationTests containing:

compileFilesInDir("path/to/the/output/files", defaultOptions)

and run it using testOnly from the sbt console.

Optionally, find compiler/src/dotty/tools/dotc/Run.scala and add:

case ex @ (_: StackOverflowError | _: ClassCastException) =>
  ctx.echo(i"exception occurred while compiling $units%, %")
  throw ex

to print more stack traces.

Using squashfs

postprocess.py supports writing results into a squashfs archive using -q flag:

python3 postprocess.py all.txt results/ -q sources.squashfs

results/ is used as a temporary directory if -q is supplied.

Credits

Code was partially derived from https://github.com/spro/char-rnn.pytorch. See char-rnn.pytorch_LICENSE for the original license.