Regex: generate matching and non-matching strings

This is a java library that, given a regex pattern, allows to:

1. Generate matching strings
2. Iterate through unique matching strings
3. Generate not matching strings

Table of contents

Status
Try it now
Usage
Supported Syntax
Configuration
Limitations
Other similar libraries
Support

Status

Build status:

Latest Release	Latest snapshot

Try it now!!!

Follow the link to Online IDE with created project: JDoodle. Enter your pattern and see the results.

Usage

Maven dependency

The Latest RELEASE:

mvnrepository.com

<dependency>
    <groupId>com.github.curious-odd-man</groupId>
    <artifactId>rgxgen</artifactId>
    <version>2.0</version>
</dependency>

Code:

Note - RgxGen is not thread safe - there were reports on errors - see #91.

public class Main {
    public static void main(String[] args) {
        RgxGen rgxGen = RgxGen.parse("[^0-9]*[12]?[0-9]{1,2}[^0-9]*");       // Create generator
        String s = rgxGen.generate();                                        // Generate new random value
        Optional<BigInteger> estimation = rgxGen.getUniqueEstimation();      // The estimation (not accurate, see Limitations) how much unique values can be generated with that pattern.
        StringIterator uniqueStrings = rgxGen.iterateUnique();               // Iterate over unique values (not accurate, see Limitations)
        String notMatching = rgxGen.generateNotMatching();                   // Generate not matching string
    }
}

public class Main {
    public static void main(String[] args) {
        RgxGen rgxGen = RgxGen.parse("[^0-9]*[12]?[0-9]{1,2}[^0-9]*");       // Create generator
        Random rnd = new Random(1234);
        String s = rgxGen.generate(rnd);                                     // Generate first value
        String s1 = rgxGen.generate(rnd);                                    // Generate second value
        String s2 = rgxGen.generate(rnd);                                    // Generate third value
        String notMatching = rgxGen.generateNotMatching(rnd);                // Generate not matching string
        // On each launch s, s1 and s2 will be the same
    }
}

Supported syntax

Configuration

RgxGen can be configured per instance.

Please refer to the following enum for all available properties: com.github.curiousoddman.rgxgen.config.RgxGenOption.

Create and Use Configuration Properties

Use new RgxGenProperties() to create properties object.

public class Main {
    public static void main(String[] args) {
        // Create properties object (RgxGenProperties extends java.util.Properties)
        RgxGenProperties properties = new RgxGenProperties();
        // Set value "20" for INFINITE_PATTERN_REPETITION option in properties
        RgxGenOption.INFINITE_PATTERN_REPETITION.setInProperties(properties, 20);
        // ... now properties can be passed to RgxGen
        RgxGen rgxGen_3 = RgxGen.parse(properties, "my-cool-pattern");
    }
}

Limitations

Dot pattern generated symbols

In regex dot . means any symbol.

By default, this would generate any value in a range defined in ASCII_SYMBOL_RANGE here com.github.curiousoddman.rgxgen.parsing.dflt.ConstantsProvider.java i.e.: any character starting from space to ~.

You can modify range of allowed values using DOT_MATCHES_ONLY configuration property.

For example:

public class Main {
    public static void main(String[] args) {
        RgxGenProperties properties = new RgxGenProperties();
        RgxGenOption.DOT_MATCHES_ONLY.setInProperties(properties, RgxGenCharsDefinition.of("abc"));
        RgxGen rgxGen = RgxGen.parse(properties, ".");
        String generatedValue = rgxGen.generate();      // Will produce either "a" or "b" or "c".
    }
}

Lookahead and Lookbehind

Currently, these two have very limited support. Please refer to #63. I'm currently working on the solution, but I cannot say when I come up with something.

Estimation

rgxGen.getUniqueEstimation() - might not be accurate, because it does not count actual unique values, but only counts different states of each building block of the expression. For example: "(a{0,2}|b{0,2})" will be estimated as 6, though actual number of unique values is 5. That is because left and right alternative can produce same value. At the same time "(|(a{1,2}|b{1,2}))" will be correctly estimated to 5, though it will generate same values.

Uniqueness

For the similar reasons as with estimations - requested unique values iterator can contain duplicates.

Infinite patterns

By design a+, a* and a{n,} patterns in regex imply infinite number of characters should be matched. When generating data, that would mean values of infinite length might be generated. It is highly doubtful anyone would require a string of infinite length, thus I've artificially limited repetitions in such patterns to 100 symbols, when generating random values. This value can be changed - please refer to configuration section.

On the contrast, when generating unique values - the number of maximum repetitions is Integer.MAX_VALUE.

Use a{n,m} if you require some specific number of repetitions. It is suggested to avoid using such infinite patterns to generate data based on regex.

Not matching values generation

The general rule is - I am trying to generate not matching strings of same length as would be matching strings, though it is not always possible. For example pattern . - any symbol - would yield empty string as not matching string. Another example a{0,2} - for this pattern not matching string would be an empty string, but I would only generate the resulting strings of 1 or 2 symbols long. I chose these approaches because they are predictable and, probably, desirable for users.

Which values are used in non-matching generation

Whenever non-matching result is requested, with either RgxGen.parse(".").generateNotMatching() method or with pattern, like "[^a-z]" - there is a choice in generator which are characters that do not match mentioned characters. For example - for "[^a-z]" - any unicode character except the ones in a range a-z would be ok. Though that would include non-printable, all kinds of blank characters and all the different wierd unicode characters. I see that as not an expected behavior. Thus, I have defined 2 different universe ranges of characters that are used - one for the ASCII only characters and another - for unicode characters.

These ranges are defined here:

• ASCII: ASCII_SYMBOL_RANGE constant in com.github.curiousoddman.rgxgen.parsing.dflt.ConstantsProvider.java
• Unicode: UNICODE_SYMBOL_RANGE constant in com.github.curiousoddman.rgxgen.parsing.dflt.ConstantsProvider.java

UNICODE_SYMBOL_RANGE is currently used ONLY when Character Classes are used \p{} ir \P{} patterns. By default ASCII_SYMBOL_RANGE is used.

To generate not matching characters I take one of the aforementioned constant ranges and subtract characters provided in pattern - resulting range is the one that is used for non-matching generation. For example for pattern "[^a-z]" ASCII_SYMBOL_RANGE will be used as a universe. The result then will be ASCII_SYMBOL_RANGE except A-z = space - @ union { - ~

Unicode Categories

Be warned - unicode categories might provide unexpectedly wrong result depending on Java version used: #99. To be absolutely sure that on your java version patterns are generated correctly I suggest running RgxGen tests with your java version.

To create categories I've used Java (corretto-17.0.10) Pattern.compile() to split characters into categories. Unfortunately there were several character categories that are not supported by Java Pattern.compile() as a result these are not currently supported.

For complete list of characters per category please refer to this directory. Each file represents one category. Each line in a file describes one symbol from the category.

Supported keys for categories can be found in com.github.curiousoddman.rgxgen.model.UnicodeCategory

Other tools to generate values by regex and why this might be better

There are 2 more libraries available to achieve same goal:

1. https://github.com/mifmif/Generex
2. http://code.google.com/p/xeger

Though I found they have the following issues:

1. All of them build graph which can easily produce OOM exception. For example pattern a{60000}, or IPV6 regex pattern.
2. Alternatives - only 2 alternatives gives equal probability of each alternative to appear in generated values. For example: (a|b) the probability of a and b is equal. For (a|b|c) it would be expected to have a or b or c with probability 33.(3)% each. Though really the probabilities are a=50%, and b=25% and c=25% each. For longer alternatives you might never get the last alternative.
3. They are quite slow
4. Lightweight. This library does not have any dependencies.

Support

I plan to support this library, so you're welcome to open issues or reach me by e-mail in case of any questions. Any suggestions, feature requests or bug reports are welcome!

Please vote up my answer on StackOverflow to help others find this library.