Statuses and verdicts

Every submission carries two separate fields, and they are routinely confused. The status says how far the evaluation got. The verdict says how the program did, and only exists for an evaluation that ran to completion.

The badge you see in a list is one field or the other: it shows the verdict once the status is Complete, and the status in every other case. So a badge reading Testing is a status — the run hasn't finished — while a badge reading Wrong answer is a verdict, and the score behind it is final. Both fields are also available as separate filters on the submission list.

Statuses

Where the evaluation got to. Any of these can be the badge you see.

Status	Meaning
Pending	Queued, waiting for a testing slot.
Provisioning	The testing environment is being prepared.
Compiling	The program is being built.
Testing	Tests are running.
Timeout	Testing itself took too long — too many tests timed out.
Complete	Testing finished. The verdict and the score are final.
Error	The program could not be evaluated: a compilation error, or a failure before any test ran.
System failure	Testing failed because of a platform error or a broken problem configuration.
Skipped	Not run, because an earlier test failed.
Blocked	Not run, because a testset it depends on failed.

System failure is not the participant's fault and does not mean their program is wrong. It usually points at the problem's testing configuration — a missing checker, a broken generator. Fix the problem, then rejudge, and the status clears.

Verdicts

How the program did, once the status is Complete.

Verdict	Meaning
Accepted	The answer was correct.
Wrong answer	The answer was incorrect.
Time limit exceeded	The program ran longer than the problem's wall time limit.
CPU exhausted	The program used more CPU time than the problem allows.
Memory overflow	The program used more memory than the problem allows.
Runtime error	The program terminated abnormally — a crash, a non-zero exit code, a fatal signal.

The limits these verdicts are measured against belong to the problem and are set in Problems → problem → Testing. The figures shown on a submission are what it used, not what it was allowed.

Testsets and tests carry the same pair

Status and verdict are not only properties of a submission. Every testset and every individual test carries its own status and its own verdict, shown the same way in the test breakdown.

That is why a breakdown can look inconsistent at first glance and still be right: a testset can be Complete with a verdict of Wrong answer while the submission as a whole is still Testing, because the remaining testsets haven't run yet.

The values aggregate upwards. A test's verdict is its own; a testset's verdict is the aggregate of its tests; the submission's verdict is the aggregate of its testsets. A submission that scored partial credit therefore still reports the verdict of the tests it did not pass, not Accepted.

Skipped and Blocked are the two statuses you'll mostly meet at this level rather than on a whole submission. They are how the scoring rules show up in the breakdown: a testset that stops at the first failure leaves the rest of its tests Skipped, and a testset whose dependency failed is Blocked without running at all. Both are described in Testsets and scoring.

What a participant is shown of all this is a separate question, settled by the testset's feedback policy — see What participants see while solving.