Tracking: Test generator MVP

[mk-mxp]

Test generator MVP to use after configlet generate for practice exercises:

• Test generator, step 1 #638 Integrate POC test generator into new directory contrib/generator, so it is not installed in CI
• Test generator, step 1 #638 Add a parameter to choose the exercise by slug
• Test generator, step 1 #638 Use local configlet cached problem specification repository as information source
• Re-write test generator with templates and unit tests #663 Do not do anything, if exercise has no canonical-data (there are some)
• Re-write test generator with templates and unit tests #663 Use /exercises/practice/${slug}/.meta/config.json>files>test and check that there is only one entry
• Test generator, step 1 #638 Write tests to already created /exercises/practice/${slug}
• Test generator, step 1 #638 Have test class skeleton
• Re-write test generator with templates and unit tests #663 Have test class DocBlock with "PHP track rules"
• Test generator, step 1 #638 Have testdox with description
• Test generator, step 1 #638 Have UUID in DocBlock
• Test generator, step 1 #638 Have all tests marked incomplete
• Re-write test generator with templates and unit tests #663 Have $input, $expected
• Re-write test generator with templates and unit tests #663 Have $subject = new ${PascalSlug};, $actual = $subject->${property}($input); and assertSame($expected, $actual);
• Re-write test generator with templates and unit tests #663 Handle unknown data, so the user can act on it
• Re-write test generator with templates and unit tests #663 Handle nested groups of tests
• Bonus: Add composer script to handle exercise generation from scratch. The problem is handling the slug required for exercise generation. That requires some additional scripting, so that only those commands get the slug that need it and the others do not fail.
• Re-write test generator with templates and unit tests #663 adds unit tests, but no integration tests Bonus: Add golden tests for test generation of some exercises
• Re-write test generator with templates and unit tests #663 handles unknow data instead Bonus: Add JSON validation using the configlet scheme (if that exists), else at least field existance and character sets
• Bonus: Add PHP CodeSniffer (with rules for test-generator only) and more static analysis

Not MVP:

• Producing production ready tests
• scenarios
• Reimplementation
• Ignoring tests marked with include = false in tests.toml

Open questions:

• Should it be a single command app or do we want to add more commands like update-exercise (or so)?
• Should we use another code producer library like laminas/laminas-code?

[mk-mxp]

@tomasnorre Did you add some functionality to your local copy of the test generator that you would want to have here, too? I think I'll find time on the weekend to take some of the steps outlined here and would be happy to integrate the "latest" version.

[tomasnorre]

@mk-mxp I don't have anything locally, that is not already published here https://github.com/tomasnorre/exercism-tests-generation