Checkers
A checker decides whether what a participant's program printed is a correct answer for one test. It runs after that program has finished, with access to the test's input, the program's output and the stored answer, and its verdict is what turns a run into a passed or failed test.
Every problem needs one. A problem with neither a checker nor an interactor cannot grade submissions at all.
The checker is set on the problem's Testing tab: the Checker row shows the current checker type, or the runtime name when the checker is a program, and opens the checker in the Studio. There you pick the type from the list at the top of the editor and save. You need permission to write problems.
A widget problem sets its checker on the Widget tab instead, and its checker must be a program.
Built-in checkers
These need nothing compiled: pick one, set its options and save.
Line by line
Compares the output and the answer line by line. Trailing whitespace on a line and empty lines at the end of the file are ignored; everything else has to match exactly.
Use it when there is exactly one acceptable output and you want it produced exactly.
Tokens
Compares token by token, so whitespace and line breaks stop mattering. Numbers are compared as numbers, to a Precision you set, and strings are compared with Case sensitive on or off as you choose.
Use it when the answer is a sequence of numbers or words and you do not want to fail a correct solution over a stray space. On very large outputs it is the slower of the two built-ins.
Query results
Compares JSON-encoded query results. Order sensitive controls whether the order of rows counts: with it off, the rows of the output and the answer are sorted before comparison. It is the checker of SQL problems, and is not offered for new problems.
None
No verification at all. Only sensible when the output is there to be looked at rather than graded.
Custom program checkers
When correctness cannot be decided by comparing text — several valid answers, an answer that has to be verified rather than matched, partial credit — write the checker as a program. Choose Custom program, then pick the runtime, add additional files if the checker needs them, and write the source.
A program checker also has a Mode, which says how the checker is called and how its result is read. Pick the one the checker was written for; a checker run in the wrong mode reads its files in the wrong order or has its verdict misread.
Mode | Called as | Result |
|---|---|---|
Eolymp |
| Read as testlib reads it. The mode for checkers written with |
Testlib |
| Exit code 0 is accepted, 1 or 2 is a wrong answer, 7 is partial credit with the points written to the log. |
CMS |
| A number from 0 to 1 printed to standard output is the share of the test's points; a message goes to standard error. |
Kattis |
| Exit code 42 is accepted, 43 is a wrong answer. |
A checker that awards part of a test's points produces the Partially correct verdict for that test. Whether partial points count towards the score is decided by the testset; see Testsets and scoring.
Checkers written for testlib work unchanged in the Eolymp and Testlib modes. eolymp.h is provided by the judge, so it needs no entry in the additional files; its reference is at eolymp.h reference.
When the checker runs
A program checker runs only for submissions that finished without a runtime error, a timeout or a memory violation. If the participant's program crashed or ran out of time or memory, that is already the verdict and the checker is never consulted — so a checker never has to cope with a truncated or missing output file.
For interactive problems the interactor runs first; the checker grades the test only if the interaction ended successfully.
Replacing a checker
Pick another type and save to replace the checker. A checker marked secret, as imported problems often are, opens in the Studio as Secret data and cannot be read.
Apoye a Ucrania