:heavy_check_mark: Automated Grading Tool that Checks the Work of Writers and Programmers
Although small, this patch release incorporates changes from three PRs that improve the visibility of information in both GatorGrader output and documentation, as well as formalize the PyPI packaging implementation. Especially important are the README.md updates to ensure correctness about the use of Poetry.
Full Changelog: https://github.com/GatorEducator/gatorgrader/compare/v1.1.0...v1.1.1
Note: This release should be viewed as newer than v1.1.0-legacy
This release refactors a significant amount of GatorGrader internals, and is the first release available on PyPI. The release was initially intended to be a stop-gap solution. In future versions, some changes may be made to the structure and distribution of the PyPI package. Additionally, the documentation available in this release may or may not be fully accurate. For users who just wish to use GatorGrader on the command line, all that is needed is to install this package and run gatorgrader. For help on syntax and available options, see gatorgrader --help.
For users who would like to interact with GatorGrader programmatically, refer to the docstrings available in gatorgrader.py and gator/__init__.py (or view them in your editor-of-choice with support for a Python Language Server). In summary, however, GatorGrader can be run by importing the gator package and calling gator.grader(["list", "--of", "command", "--line", "arguments"]), the syntax of which matches previous versions of GatorGrader. For additional help or guidance, see gatorgrader --help or reach out to any of the core maintainers.
Now, the optional --description argument allows setting a custom description per check, to cut down on student confusion. In addition, this release includes changes to the release workflow and continuous integration setup.
Note: This release is the last one compatible with GatorGradle Note: This release is compatible with Python 3.7+
Full Changelog: https://github.com/GatorEducator/gatorgrader/compare/v1.0.5...v1.1.0-legacy
This PR is a bugfix that resolves issues with incorrect code formatting and the setup with AppVeyor and Travis CI.
@corlettim: This PR is to help with fixing the issued line ending error that we have come upon.
@gkapfham: This PR includes the following modifications:
.gitattributes file that mandates that line endings follow the Unix standardPipfile.lock
Auto-generated by pr-tag-release
This PR is a bugfix that resolves issues with incorrect code formatting and the setup with AppVeyor and Travis CI.
@corlettim: This PR is to help with fixing the issued line ending error that we have come upon.
@gkapfham: This PR includes the following modifications:
.gitattributes file that mandates that line endings follow the Unix standardPipfile.lock
Auto-generated by pr-tag-release
This PR is a bugfix that resolves issues with incorrect code formatting and the setup with AppVeyor and Travis CI.
@corlettim: This PR is to help with fixing the issued line ending error that we have come upon.
@gkapfham: This PR includes the following modifications:
.gitattributes file that mandates that line endings follow the Unix standardPipfile.lock
Auto-generated by pr-tag-release
Updated to say "The repository has..." rather than "Repository has..." to match other content.
Check displays text "Repository has" which does not match current format (i.e. it is missing a preceding "the" to start the statement). This PR is associated with the below issue:
https://github.com/GatorEducator/gatorgrader/issues/194
Updated to say "The repository has..." to match current format
This PR is a small change that fixes #195
@ajciancimino
Auto-generated by pr-tag-release
Updated to say "The repository has..." rather than "Repository has..." to match other content.
Check displays text "Repository has" which does not match current format (i.e. it is missing a preceding "the" to start the statement). This PR is associated with the below issue:
https://github.com/GatorEducator/gatorgrader/issues/194
Updated to say "The repository has..." to match current format
This PR is a small change that fixes #195
@ajciancimino
Auto-generated by pr-tag-release
This PR adopts a plugin-based approach to provide a linter-inspired interface.
This means:
-- A person using GatorGrader will be able to introduce their own checks dynamically -- The command-line interface will feature positional required arguments for all checks
This PR also introduces a significant number of bug fixes and improvements to the tool. For instance, there were a number of problems in the way that GatorGrader previously handled the combination of wildcards with the use of --exact. These issues have been resolved, at the expense of making the code more complex. I suggest that we still merge this PR and then revisit this complexity in a future PR. For instance, the invoke_all_comment_checks function in the invoke module is now more complex that is appropriate, in part because it now correctly makes diagnostics for a wide variety of edge cases.
GatorGrader currently does not:
-- Allow a person to specify their own checks. As such, the tool is currently not extensible. -- Allow the specification of checks in an easy-to-document and easy-to-use fashion.
GatorGrader will now support a plugin-based and linter-inspired interface. It does so by using the PluginBase package described at: https://github.com/mitsuhiko/pluginbase
@jjumadinova and @amohangit and @dluman and @obonhamcarter please note that once this PR is merged, the way in which you will write checks in a gatorgrader.yml file will change. If you want to continue to use the old approach, you can use a version: tag in the YAML header of your gatorgrader.yml file, using a method that @Michionlion will finish implementing before the Fall 2019 semester starts. With that said, @Michionlion and all of the other core maintainers of GatorGrader suggest that you adopt this new approach instead as it will be easier to specify your own checks and add checks that are not currently provided by GatorGrader.
If @jjumadinova or @amohangit or @dluman or @obonhamcarter have questions about the implications of merging this PR to GatorGrader, they should communicate with @gkapfham.
There are currently a few branches and lines of code that are not covered by the unit test test suite. With that said, this PR has 99% coverage of the statements and the branches and I think that this is high enough to support the merge of the PR.
Here are the details from the coverage report:
Test session starts (platform: linux, Python 3.7.3, pytest 5.0.1, pytest-sugar 0.9.2)
rootdir: /home/gkapfham/working/source/gatorgrader, inifile: pytest.ini
plugins: sugar-0.9.2, cov-2.7.1
collecting ...
tests/test_arguments.py ✓✓✓✓✓✓✓✓✓✓ 1% ▎
tests/test_checkers.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 4% ▍
tests/test_comments.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 17% █▊
✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 20% ██
tests/test_constants.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 24% ██▌
tests/test_display.py ✓✓✓✓✓✓ 25% ██▌
tests/test_files.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 27% ██▊
tests/test_fragments.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 40% ████
✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 46% ████▋
tests/test_invoke.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 50% █████
tests/test_leave.py ✓✓✓✓✓✓✓✓✓✓ 51% █████▏
tests/test_markdown.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 53% █████▍
tests/test_orchestrate.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 55% █████▌
tests/test_report.py ✓✓✓✓✓✓ 56% █████▋
tests/test_repository.py ✓✓✓✓✓✓✓✓✓✓✓ 57% █████▊
tests/test_run.py ✓✓✓✓✓✓ 58% █████▊
tests/test_util.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 63% ██████▍
tests/checks/test_check_ConfirmFileExists.py ✓✓✓✓✓✓✓✓✓✓✓ 64% ██████▍
tests/checks/test_check_CountCommandOutput.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 66% ██████▋
tests/checks/test_check_CountCommits.py ✓✓✓✓✓✓✓✓✓✓✓ 67% ██████▊
tests/checks/test_check_CountFileLines.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 70% ██████▉
tests/checks/test_check_CountFileParagraphs.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 72% ███████▎
tests/checks/test_check_CountFileWords.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 74% ███████▍
tests/checks/test_check_CountMarkdownTags.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 77% ███████▋
tests/checks/test_check_CountMultipleLineComments.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 80% ████████
tests/checks/test_check_CountParagraphWords.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 82% ████████▎
tests/checks/test_check_CountSingleLineComments.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 86% ████████▋
tests/checks/test_check_ExecuteCommand.py ✓✓✓✓✓✓✓✓✓✓ 87% ████████▊
tests/checks/test_check_ListChecks.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 89% ████████▉
tests/checks/test_check_MatchCommandFragment.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 92% █████████▎
tests/checks/test_check_MatchCommandRegex.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 95% █████████▌
tests/checks/test_check_MatchFileFragment.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 98% █████████▊
tests/checks/test_check_MatchFileRegex.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 100% ██████████
----------- coverage: platform linux, python 3.7.3-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing
---------------------------------------------------------------------------------------------
gator/__init__.py 0 0 0 0 100%
gator/arguments.py 31 0 4 0 100%
gator/checkers.py 84 0 36 0 100%
gator/checks/__init__.py 0 0 0 0 100%
gator/checks/check_ConfirmFileExists.py 16 0 0 0 100%
gator/checks/check_CountCommandOutput.py 19 0 0 0 100%
gator/checks/check_CountCommits.py 18 0 0 0 100%
gator/checks/check_CountFileLines.py 22 0 0 0 100%
gator/checks/check_CountFileParagraphs.py 21 0 0 0 100%
gator/checks/check_CountFileWords.py 23 0 0 0 100%
gator/checks/check_CountMarkdownTags.py 24 0 0 0 100%
gator/checks/check_CountMultipleLineComments.py 24 0 0 0 100%
gator/checks/check_CountParagraphWords.py 23 0 0 0 100%
gator/checks/check_CountSingleLineComments.py 24 0 0 0 100%
gator/checks/check_ExecuteCommand.py 14 0 0 0 100%
gator/checks/check_ListChecks.py 28 0 4 0 100%
gator/checks/check_MatchCommandFragment.py 21 0 0 0 100%
gator/checks/check_MatchCommandRegex.py 21 0 0 0 100%
gator/checks/check_MatchFileFragment.py 24 0 0 0 100%
gator/checks/check_MatchFileRegex.py 24 0 0 0 100%
gator/comments.py 26 0 0 0 100%
gator/constants.py 25 0 0 0 100%
gator/display.py 20 0 0 0 100%
gator/entities.py 29 0 6 0 100%
gator/files.py 39 0 8 0 100%
gator/fragments.py 114 0 40 0 100%
gator/invoke.py 200 4 80 4 96% 186->202, 193->197, 197->198, 198-201, 330->336
gator/leave.py 5 0 2 0 100%
gator/markdown.py 21 0 6 0 100%
gator/orchestrate.py 67 0 14 0 100%
gator/report.py 34 0 2 0 100%
gator/repository.py 24 0 2 0 100%
gator/run.py 25 0 4 0 100%
gator/util.py 145 0 54 0 100%
---------------------------------------------------------------------------------------------
TOTAL 1235 4 262 4 99%
Results (11.27s):
922 passed
This PR was tested with an "integration test suite" involving a series of checks in a gatorgrader.yml file that is included below. Please note that all of these checks are designed to pass correctly. There are also some comments in the file that explain the types of checks that would not, in fact, pass correctly.
---
name: cmpsc-203-spring-2019-practical3
break: true
indent: 4
revision: feature/use-linters-plugins
---
# --> check the source code for various characteristics
# note that without an \"--exact\" the check is an \"at least\"
termfrequency:
compute_tf_cookbook.py:
ConfirmFileExists
CountFileLines --count 10
CountFileLines --count 86 --exact
MatchFileFragment --fragment \"TODO\" --count 0 --exact
MatchFileFragment --fragment \"for tf in word_freqs\" --count 1
MatchFileFragment --fragment \"word_freqs\" --count 1
CountSingleLineComments --language Python --count 12
CountMultipleLineComments --language Python --count 7 --exact
tests:
test_compute_tf_cookbook.py:
ConfirmFileExists
MatchFileFragment --fragment \"from termfrequency import compute_tf_cookbook\" --count 1
MatchFileFragment --fragment \"test_\" --count 5
CountFileLines --count 20
termfrequency:
*.py:
MatchFileFragment --fragment \"TODO\" --count 0 --exact
CountFileLines --count 0
tests:
*.py:
MatchFileFragment --fragment \"TODO\" --count 0 --exact
CountFileLines --count 0
# --> check the technical writing, specify a specific file
# mdl is a Markdown linting tool
# proselint checks writing for common mistakes
writing/reflection.md:
mdl
proselint
ConfirmFileExists
CountParagraphWords --count 100
CountParagraphWords --count 100 --exact
CountFileWords --count 200
CountFileWords --count 600 --exact
MatchFileFragment --fragment \"```\" --count 2 --exact
MatchFileRegex --regex \"```\" --count 2 --exact
MatchFileRegex --regex \"## Explain [.]*\" --count 4 --exact
CountMarkdownTags --tag \"heading\" --count 8 --exact
CountMarkdownTags --tag \"code\" --count 3 --exact
CountMarkdownTags --tag \"code_block\" --count 1 --exact
CountFileParagraphs --count 6 --exact
# --> check the technical writing, specify a specific file
# Note that third-party tools like mdl and proselint may not work with
# wildcards and thus, since they are not an official part of GatorGrader
# they cannot be specified with a wildcard filename.
# Also note that in this case there is only one file in the writing/
# directory and thus the checks are exactly the same as the prior block.
writing/*.md:
CountParagraphWords --count 100
CountParagraphWords --count 100 --exact
CountFileWords --count 200
CountFileWords --count 600 --exact
MatchFileFragment --fragment \"```\" --count 2 --exact
MatchFileRegex --regex \"```\" --count 2 --exact
MatchFileRegex --regex \"## Explain [.]*\" --count 4 --exact
CountMarkdownTags --tag \"heading\" --count 8 --exact
CountMarkdownTags --tag \"code\" --count 3 --exact
CountMarkdownTags --tag \"code_block\" --count 1 --exact
CountFileParagraphs --count 6 --exact
# # --> check the number of commits beyond a threshold
CountCommits --count 10
# --> check that the program executes and produces 4 lines of output
ExecuteCommand --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\"
CountCommandOutput --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 8 --exact
MatchCommandFragment --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 0 --fragment \"the\" --exact
MatchCommandFragment --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 1 --fragment \"live\" --exact
MatchCommandFragment --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 6 --fragment \"1\" --exact
MatchCommandRegex --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 0 --regex \"(the)+\" --exact
MatchCommandRegex --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 1 --regex \"(live)+\" --exact
MatchCommandRegex --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 6 --regex \"1\" --exact
# this would not work correctly because it matches everything
# MatchCommandRegex --command \"pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt\" --count 1 --regex \"(the)*\" --exact
# --> check that the test suite executes and does not fail
ExecuteCommand --command \"pipenv run pytest\"
ExecuteCommand --command \"pipenv run pytest -x -s --cov-config pytest.cov --cov-report term-missing --cov\"
Here is the output from running these checks:
gkapfham in solutions/cs203-S2019-practical3-solution masterU8 C6 gradle grade
> Configure project :
Configured GatorGradle 0.4.3
> Task :grade
Updating GatorGrader...
Fetching origin
Checking out to 'feature/use-linters-plugins'
Managing GatorGrader's Python dependencies...
Finished!
✔ Repository has at least 10 commit(s)
✔ The reflection.md in writing has exactly 3 of the 'code' tag
✔ The *.md in writing has exactly 1 of the 'code_block' tag
✔ The command output has exactly 0 match(es) of the '(the)+' regular expression
✔ The command output has exactly 6 match(es) of the '1' regular expression
✔ The command output has exactly 0 of the 'the' fragment
✔ The file writing/reflection.md passes proselint
✔ The *.py in tests has exactly 0 of the 'TODO' fragment
✔ The *.md in writing has exactly 2 match(es) of the '```' regular expression
✔ The test_compute_tf_cookbook.py in tests has at least 5 of the 'test_' fragment
✔ The compute_tf_cookbook.py in termfrequency has at least 1 of the 'for tf in word_freqs' fragment
✔ The *.md in writing has exactly 8 of the 'heading' tag
✔ The test_compute_tf_cookbook.py in tests has at least 1 of the 'from termfrequency import compute_tf_cookbook' fragment
✔ The reflection.md in writing has at least 100 word(s) in every paragraph
✔ The reflection.md in writing has exactly 600 word(s) in total
✔ The file writing/reflection.md passes mdl
✔ The reflection.md in writing has exactly 2 match(es) of the '```' regular expression
✔ The command output has exactly 6 of the '1' fragment
✔ The test_compute_tf_cookbook.py in tests has at least 20 line(s)
✔ The *.md in writing has exactly 600 word(s) in total
✔ The compute_tf_cookbook.py in termfrequency has exactly 86 line(s)
✔ The *.md in writing has at least 200 word(s) in total
✔ The reflection.md in writing has exactly 1 of the 'code_block' tag
✔ The command 'pipenv run pytest -x -s --cov-config pytest.cov --cov-report term-missing --cov' executes correctly
✔ The reflection.md in writing has exactly 4 match(es) of the '## Explain [.]*' regular expression
✔ The command 'pipenv run pytest' executes correctly
✔ The file test_compute_tf_cookbook.py exists in the tests directory
✔ The reflection.md in writing has at least 200 word(s) in total
✔ The reflection.md in writing has exactly 8 of the 'heading' tag
✔ The file reflection.md exists in the writing directory
✔ The *.md in writing has at least 100 word(s) in every paragraph
✔ The reflection.md in writing has exactly 6 paragraph(s)
✔ The command output has exactly 8 lines
✔ The *.py in termfrequency has at least 0 line(s)!
✔ The *.py in tests has at least 0 line(s)
✔ The *.md in writing has exactly 2 of the '```' fragment
✔ The *.md in writing has exactly 6 paragraph(s)
✔ The compute_tf_cookbook.py in termfrequency has exactly 0 of the 'TODO' fragment
✔ The command 'pipenv run python3 termfrequency/compute_tf_cookbook.py inputs/input.txt' executes correctly
✔ The compute_tf_cookbook.py in termfrequency has at least 12 single-line Python comment(s)
✔ The reflection.md in writing has exactly 2 of the '```' fragment
✔ The file compute_tf_cookbook.py exists in the termfrequency directory
✔ The compute_tf_cookbook.py in termfrequency has at least 1 of the 'word_freqs' fragment
✔ The command output has exactly 1 of the 'live' fragment
✔ The reflection.md in writing has exactly 100 word(s) in every paragraph
✔ The *.md in writing has exactly 3 of the 'code' tag
✔ The *.md in writing has exactly 4 match(es) of the '## Explain [.]*' regular expression
✔ The *.md in writing has exactly 100 word(s) in every paragraph
✔ The compute_tf_cookbook.py in termfrequency has at least 10 line(s)
✔ The *.py in termfrequency has exactly 0 of the 'TODO' fragment
✔ The compute_tf_cookbook.py in termfrequency has exactly 7 multiple-line Python comment(s)
✔ The command output has exactly 1 match(es) of the '(live)+' regular expression
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Passed 52/52 (100%) of checks for cmpsc-203-spring-2019-practical3! ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
BUILD SUCCESSFUL in 14s
1 actionable task: 1 executed
The following checks were extensively testing when they were run manually by @gkapfham:
Please note that the checks called CountCommandOutput, MatchCommandFragment, and MatchCommandRegex do not accept wildcard inputs because they are related to checking the output of commands run on the computer. The remaining checks were manually run with both wildcards and without wildcards. The exact commands are available upon request.
The specific circumstances for the manual testing of each check are as follows:
--exact parameter and:
--exact parameter and:
After perform the manual testing for all of the aforementioned circumstances, @gkapfham determined that the tool is working correctly with the source code in this PR. Please bear in mind that all of this manual testing took place while @gkapfham directly used the tool by typing the gradle grade command. For your reference, here are all of the checks currently supported by GatorGrader if this PR is merged:
gkapfham in source/gatorgrader feature/use-linters-plugins ✔ pipenv run python gatorgrader.py ListChecks
✔ GatorGrader: Automatically Check the Files of Programmers and Writers
https://github.com/GatorEducator/gatorgrader
✔ Find the available checks that match an optional pattern
usage: ConfirmFileExists [-h] --file FILE --directory DIR
Check Provided by GatorGrader: ConfirmFileExists
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
usage: CountCommandOutput [-h] --command COMMAND --count COUNT [--exact]
Check Provided by GatorGrader: CountCommandOutput
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--command COMMAND command to execute (default: None)
--count COUNT how many of an entity should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: CountCommits [-h] --count COUNT [--exact]
Check Provided by GatorGrader: CountCommits
optional arguments:
-h, --help show this help message and exit
required check arguments:
--count COUNT minimum number of git commits (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: CountFileLines [-h] --file FILE --directory DIR --count COUNT [--exact]
Check Provided by GatorGrader: CountFileLines
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--count COUNT how many lines should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: CountFileParagraphs [-h] --file FILE --directory DIR --count COUNT
[--exact]
Check Provided by GatorGrader: CountFileParagraphs
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--count COUNT how many lines should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: CountFileWords [-h] --file FILE --directory DIR --count COUNT [--exact]
Check Provided by GatorGrader: CountFileWords
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--count COUNT how many total words should exist in the file (default:
None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: CountMarkdownTags [-h] --tag TAG --file FILE --directory DIR --count
COUNT [--exact]
Check Provided by GatorGrader: CountMarkdownTags
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--tag TAG markdown tag that exists in a file (default: None)
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--count COUNT how many tag instances should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
examples of available tags: code, heading, image, link, list, paragraph
reference: https://spec.commonmark.org/0.29/
usage: CountMultipleLineComments [-h] --file FILE --directory DIR --count
COUNT [--language {Java,Python}] [--exact]
Check Provided by GatorGrader: CountMultipleLineComments
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--count COUNT how many lines should exist (default: None)
--language {Java,Python}
language for the comments (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: CountParagraphWords [-h] --file FILE --directory DIR --count COUNT
[--exact]
Check Provided by GatorGrader: CountParagraphWords
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--count COUNT how many words should exist in every paragraph (default:
None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: CountSingleLineComments [-h] --file FILE --directory DIR --count COUNT
[--language {Java,Python}] [--exact]
Check Provided by GatorGrader: CountSingleLineComments
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--count COUNT how many comments should exist (default: None)
--language {Java,Python}
language for the comments (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: ExecuteCommand [-h] --command COMMAND
Check Provided by GatorGrader: ExecuteCommand
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--command COMMAND command to execute (default: None)
usage: ListChecks [-h] [--namecontains LABEL]
Check Provided by GatorGrader: ListChecks
optional arguments:
-h, --help show this help message and exit
optional check arguments:
--namecontains LABEL filter by label that name must contain (default: None)
usage: MatchCommandFragment [-h] --command COMMAND --fragment FRAGMENT --count
COUNT [--exact]
Check Provided by GatorGrader: MatchCommandFragment
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--command COMMAND command to execute (default: None)
--fragment FRAGMENT fragment that exists in command output (default: None)
--count COUNT how many of an entity should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: MatchCommandRegex [-h] --command COMMAND --regex REGEX --count COUNT
[--exact]
Check Provided by GatorGrader: MatchCommandRegex
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--command COMMAND command to execute (default: None)
--regex REGEX regular expression that matches command output (default:
None)
--count COUNT how many of an entity should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: MatchFileFragment [-h] --file FILE --directory DIR --fragment FRAGMENT
--count COUNT [--exact]
Check Provided by GatorGrader: MatchFileFragment
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--fragment FRAGMENT fragment that exists in the file (default: None)
--count COUNT how many of an entity should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
usage: MatchFileRegex [-h] --file FILE --directory DIR --regex REGEX --count
COUNT [--exact]
Check Provided by GatorGrader: MatchFileRegex
optional arguments:
-h, --help show this help message and exit
required checker arguments:
--file FILE file for checking (default: None)
--directory DIR directory with file for checking (default: None)
--regex REGEX regular expression that matches file contents (default:
None)
--count COUNT how many of an entity should exist (default: None)
optional check arguments:
--exact equals instead of a minimum number (default: False)
The ideas for this PR were discussed with @Michionlion and @alexheinle and @corlettim and @schultzh. Please note that great care was taken to ensure that the modules of GatorGrader were better tested at the unit level; previously some of the functions were tested indirectly through invoke or orchestrate functions and this make understanding and improving code coverage and the tests much more difficult.
During the development of this PR, it was apparent that there is some "under-specified" behavior in GatorGrader that resulted from the merge of previous PRs.
In the following example, the check is to see if an incorrectly specified file has 0 single-line comments. The informal GatorGrader specification does not indicate whether or not this check should pass. Since the file does not exist, if cannot, by definition, have any single-line comments inside of it. However, the check is requiring, for instance, that it has exactly 0 single-line comments — which, in a way, a non-existing file "does" have!
pipenv run python gatorgrader.py CountSingleLineComments --file \"test_file_WRONG*.py\" --directory tests --count 0 --language Python --exact
✔ GatorGrader: Automatically Check the Files of Programmers and Writers
https://github.com/GatorEducator/gatorgrader
✔ The test_file_WRONG*.py in tests has exactly 0 single-line Python comment(s)
The current implementation of GatorGrader will pass this check. However, we may wish to revisit this issue in the future so that we can better specify the intended behavior. I suggest that it is acceptable to merge this PR with the issue remaining unresolved because it is a case that is unlikely to manifest.
This PR is a compatibility breaking update that fixes #175.
@gkapfham
Auto-generated by pr-tag-release
This pull request deletes mutmut from the Pipfile and Pipfile.lock since we are not doing mutation testing on the test suite.
Currently, mutmut is a dependency.
See issue #183 for more details.
This PR will delete the mutmut dependency
This PR is a small change that fixes #183
@MaddyKapfhammer
Auto-generated by pr-tag-release