With the RosettaTest project, you only have to write a test suite once for multiple implementations.
(This is still a prototype project. Expect major architectural or API changes, as well as major bugs and inconsistencies.)
Currently, the best-supported language is Python. To get an idea of how to create and port RosettaTest suites, try the Python version of the SMTP test suite, which runs against the SMTP implementation in the Python standard library.
You can run the SMTP test suite in Python by executing:
python3 rosetta-test-py/smtp.pyThis will run the test suite from /rosetta-test-suites/smtp.rosetta using the mappings in /rosetta-test-py/smtp.py.
- The core RosettaTest language (
/rosetta-test) - RosettaTest language interpreters for
- Python (
/rosetta-test-py) - Ruby (
/rosetta-test-rb) - JavaScript (
/rosetta-test-js) - Smalltalk (
/rosetta-test-s)
- Python (
- A full test suite (
/rosetta-test-suites) for- SMTP (
smtp.rosetta) - RFC JSON parsing (
json-rfc.rosetta)
- SMTP (
- Prototype test suites (
/rosetta-test-suites) for- sending MIME documents (
sendmail.rosetta) - RFC URI parsing (
url-parsing-rfc.rosetta)
- sending MIME documents (
The test suites are written in the RosettaTest language. The parts that are implementation-specific are designated as placeholders. For every implementation that you want to test, you need to fill in these placeholders.
Thus, if you want to execute a RosettaTest suite for your project, you need:
- A RosettaTest language interpreter for your language
- A mapping that fills each placeholder with a function from the implementation. Often the mapping will not be 1:1, so you might need to write additional code to map the behavior to the expected behavior of the placeholder.
- Syntax tests based on the corpus from tree-sitter-scheme
- JSON-RFC Suite tests are based on the JSONTestSuite
- Miksus/red-mail#96 (proposed)
- lavr/python-emails#173 (proposed)
- python/cpython#132797 (proposed)
- Squeak/Smalltalk SMTPClient Patch (proposed)