Statuses and verdicts

Every submission carries two separate fields, and they are routinely confused. The status says how far the evaluation got. The verdict says how the program did, and only exists for an evaluation that ran to completion.

The badge you see in a list is one field or the other: it shows the verdict once the status is Complete, and the status in every other case. So a badge reading Testing is a status — the run hasn't finished — while a badge reading Wrong answer is a verdict, and the score behind it is final. Both fields are also available as separate filters on the submission list.

Statuses

Where the evaluation got to. Any of these can be the badge you see.

Status	Meaning
Pending	Queued, waiting for a testing slot.
Provisioning	The testing environment is being prepared.
Compiling	The program is being built.
Testing	Tests are running.
Timeout	Testing itself took too long — too many tests timed out.
Complete	Testing finished. The verdict and the score are final.
Error	The program could not be evaluated: a compilation error, or a failure before any test ran.
System failure	Testing failed because of a platform error or a broken problem configuration.
Skipped	Not run, because an earlier test failed.
Blocked	Not run, because a testset it depends on failed.

System failure is not the participant's fault and does not mean their program is wrong. It usually points at the problem's testing configuration — a missing checker, a broken generator. Fix the problem, then rejudge, and the status clears.

Verdicts

How the program did, once the status is Complete.

Verdict	Meaning
Accepted	The answer was correct.
Partially correct	The answer earned part of the test's points. The checker decides how many.
Wrong answer	The answer was incorrect.
Time limit exceeded	The program ran longer than the problem's time limit.
Idleness limit exceeded	The program stayed within its time limit but was stopped by the idleness limit, usually because it was waiting rather than computing.
Memory overflow	The program used more memory than the problem allows.
Runtime error	The program terminated abnormally — a crash, a non-zero exit code, a fatal signal.

A problem has two time limits, both set in Testing → Configuration. The Time limit is the one a solution is judged against, measured in CPU time. The Idleness limit is a wall-clock guard for a program that stops using the CPU, for example one blocked waiting for input; when it is left empty, it is twice the time limit and at least five seconds. A problem with only an idleness limit, as many older problems have, is judged against that limit in wall time and reports Time limit exceeded.

The figures shown on a submission are what it used against the limit it was judged by.

Testsets and tests carry the same pair

Status and verdict are not only properties of a submission. Every testset and every individual test carries its own status and its own verdict, shown the same way in the test breakdown.

That is why a breakdown can look inconsistent at first glance and still be right: a testset can be Complete with a verdict of Wrong answer while the submission as a whole is still Testing, because the remaining testsets haven't run yet.

The values aggregate upwards. A test's verdict is its own; a testset's verdict is the aggregate of its tests; the submission's verdict is the aggregate of its testsets. A submission that scored partial credit therefore still reports the verdict of the tests it did not pass, not Accepted.

Skipped and Blocked are the two statuses you'll mostly meet at this level rather than on a whole submission. They are how the scoring rules show up in the breakdown: a testset that stops at the first failure leaves the rest of its tests Skipped, and a testset whose dependency failed is Blocked without running at all. Both are described in Testsets and scoring.

What a participant is shown of all this is a separate question, settled by the testset's feedback policy — see What participants see while solving.