Jflex Versions Save

JFlex 1.9.1 is a small bugfix release:

• fix negated char classes with overlapping content (#1065, #1066): Expressions such as [^\s\n] were translated incorrectly, because \s and \n overlap and the negation operator ^ in version 1.9.0 did not handle this case correctly.

JFlex 1.9.0 is a feature release with slight performance improvements in the scanning engine. It also fixes all known bugs of version 1.8.2.

New features

• add optional token limit size (#1045, #197)
• warn for unicode escapes that are too long (#1042)
• directives and command line switches for enabling/disabling JFlex warnings (#1020)
• allow custom annotations such as @SuppressWarnings (#1027, #762)
• increase maximum supported DFA size to about 2^32 states (#1003, #952)
• better error reporting for character classes (#996, #888, #939)
• allow comments after %yylexthrow (#794)
• make maven build reproducible (#765)

Fixed bugs

• make character classes more robust (#985, #974, #986)
• fix NullPointerException when generating dot files (#769)

Other

• update dependencies
• more and better property-based tests (#953)
• use SPDX license identifiers
• avoid use of createTempDir() (was used in tests only)
• migrate CI to GitHub actions
• migrate most tests to bazel
• improve test suite to work on non-US locales
• fix example file name for minijava example
• improved simple.flex example

More comprehensive list in milestone 1.9.0

JFlex 1.8.2 is a small bugfix release. There are no new features.

• fix bug that prevented %7bit scanners from being generated (#756)
• fix %eof{ and %eofthrow{ code generation (#743)
• updated bazel build rule

More detailed list of changes in milestone 1.8.2

JFlex 1.8.1 is a small maintenance release. There are no new features or bug fixes. The only change is

• in dependency management for the CUP parser generator and runtime to re-enable building from source in the release package (#734)

More detailed list of changes in milestone 1.8.1

• yychar type has been changed from int to long in order to support large files (> 2GB) (#605)
• Add @SuppressWarnings("FallThrough") on generated lexer #454
• Defend against spoon-feeding readers not fully populating the scanning buffer #543
• Add support for Unicode 10.0 #540 11.0 #555 12.0 #556 and 12.1 #563
• Unicode Emoji properties are supported for Unicode versions 8.0+ (#546)
• Significantly decreased memory usage for unicode scanners from ~4MB to typical ~20kB. (#697)
• Macro expressions in character classes are now allowed (#216, #654)
• Expose yyatEOF() in generated scanner API (#644)
• Pipe action | now works for <<EOF>> (#201)
• Explicitly use UTF-8 encoding for skeleton files and dot files (#470)
• Maven plugin now correctly checks #include file time stamp (#694)
• Slightly optimised character classes when ^ operator is used (#682)
• Normalised character class order. This has no influence on how text is matched, but makes --dump output more comparable. (#650)
• Fixed a bug in the negation ! operator that in rare circumstances would match not everything covered by the negation (#567).
• The . expression now does not match unpaired surrogates, since these are not characters. (#544)
• Example specs now with build for ant, make, and maven
• Introduced a code LexGenerator API. #428 #448
• Add the jflex source in generated code #371 #399
• Code cleanup
  • modularisation effort
  • Removed dead code class CharSet #480
  • Use @AutoValue #505
  • Fixed PMD violations #413 #418
  • Use Truth in tests #365 #660
  • Replace commons-io by guava #319
• Dep updates
  • Updated maven dependencies #409
  • Updated the Maven wrapper to 0.4.2 #382
• Build system
  • retired ant build #432
  • now supporting Bazel build

See all changes in milestone 1.8.0

Released 2014-03-23

• the "switch" and "table" code generation options are deprecated and will be removed in JFlex 1.6
• the JFlex license has been changed from GPL to BSD.
• updated JFlex to CUP version 0.11a.
• changed the build from Ant to Maven. 523d7a9
• JFlex now mostly conforms with Unicode Regular Expressions UTS#18 Basic Unicode Support - Level 1. Supplementary code points (above the Basic Multilingual Plane) are not yet supported.
• new meta characters supported: \s, \S, \d, \D, \w, \W.
• nested character sets now supported, e.g. [[[ABC]D]E[FG]]
• new character set operations supported: union (e.g. [A||B]), intersection (e.g. [A&&B]), set-difference (e.g. [A--B]), and symmetric difference (e.g. [A~~B]).
• the meaning of the dot (".") meta character has been changed from [^\n] to [^\n\r\u000B\u000C\u0085\u2028\u2029]. Use the new --legacydot option to cause "." to be interpreted as [^\n].
• new \R meta character matches any newline: "\r\n" | [\n\r\u000B\u000C\u0085\u2028\u2029].
• new option --noinputstreamctor to not include an InputStream constructor in the generated scanner.
• %include can now be used in the rules section (#117)
• yychar and zzAtBOL should be reset for nested input streams (#107 & #108 )
• fixed bug #109 (could not match input for empty string matches.)
• fixed bug #112 & #119 (properly update zzFin when reallocating zzBuffer)
• fixed bug #115 (noncompileable scanner generation when default locale is Turkish)
• fixed bug #114 (zzEOFDone not included with pushed nested stream state)
• fixed bug #105 (can't build examples/java/)
• fixed bug #106 (impossible char class range should trigger syntax error)

• Prerequisites
  • Compilation requires jdk7 and Maven 3.5.2
  • Execution requires jdk7 and Maven 3.0
  • Compilation of generated code requires jdk 5
• CUP upgraded to 0.11b
• Option --inputstreamctor has been removed (#195)
• Code health
  • Codebase has valid doclint (#206)
  • Maven plugins update to use Java annotations rather than javadoc at-clauses.
• jflex --version or --info or --help now exits with error code 0 (#194)
• Unicode 8.0 and 9.0 are supported (#209)
• documentation improvements (#152, #187, #215, #290)
• added an --encoding option to specify input/output encoding (#164)
• make jflex start script robust for other locales (#251)
• report character position when %debug and %char are present (#207)

See https://github.com/jflex-de/jflex/milestone/10

Released 2014-06-21

• Unicode 7.0 is supported.
• In %unicode mode, supplementary code points are now handled properly.
  • Regular expressions are now code-point based, rather than code-unit/ char based.
  • Input streams are read as code point sequences - properly paired surrogate code units are read as a single character.
  • All supported Unicode properties now match supplementary characters when Unicode 3.0 or above is specified, or when no version is specified, causing the default Unicode version, Unicode 7.0 in this release, to be used.
• New \u{...} escape sequence allows code points (and whitespace-separated sequences of code points) to be specified as 1-6 hexadecimal digit values.
• Characters in matches printed in %debug mode are now Unicode escaped (\uXXXX) when they are outside the range 32..127.
• detect javadoc class comment when followed by annotation(s) (#128)
• removed the "switch" and "table" code generation options
• Option --noinputstreamctor deprecated. By default no InputStream constructor is included in the generated scanner. The capability to include one is deprecated and will be removed in JFlex 1.7.

Released 2015-03-16

1.6.1 is a maintenance release, fixing all known defects.

Changelog:

• JFlex development, wiki, and issue tracker moved to https://github.com/jflex-de/
• Fixed issue #130, "in caseless mode, chars in regexps not accepted caselessly": Caseless option works again as intended.
• Fixed issue #131, "re-enable scanning interactively or from a network byte stream": JFlex now throws an IOException when a Reader returns 0 characters.
• New example, shows how to deal with Readers that return 0 characters.
• Command line scripts work again in repository version (contributed by Emma Strubell)
• New options --warn-unused and --no-warn-unused that control warnings about unused macros.
• Fixed issue #125: %apiprivate and %cup2 switches now no longer incompatible
• Fix issue #133, "Error in skeleton.nested": Empty-string matches were taking precedence over EOF and caused non-termination. Now EOF is counted as the highest-priority empty match.
• New warning when an expression matches the empty string (can lead to non-termination).