Skip to content

Challenge submission

To really participate in the challenge and to have a shot at winning the prize, you will need to submit your results. There will be two ways to do that:

Rolling evaluation of agents

If you have registered with us, you were sent an API key that you can use to submit your results for evaluation. To do this, either create a file named api_key in the challenge directory and paste the key there. Or select the Execution menu from the AICA challenge TUI:

image

Then the New evaluation:

image

And if the api_key file is not there, you will be prompted to enter the key:

image

Once this is taken care of, you can select the scenario which you want to evaluate:

image

And the agent:

image

If you choose the agent, you will be prompted to confirm download of all scenarios. Be aware that the evaluation is always done on freshly downloaded scenarios, so your modifications do not propagate to the scoreboard. If you have some, please, cancel the process, backup your changes, and then restart the process anew...

image

If you got this far, you will have to wait until the evaluation runs are finished. The final output may look like this:

image

If your score is high enough, you should see it in a moment at the scoreboard.

Final evaluation

Before the deadline of 24. 10. 2025 you can submit your agent for final evaluation. Due to potentially involved complexities, the system does not provide an automated way to submit your code.

So, to submit an agent for final evaluation, please use any reasonable sharing service and send a link to aica-challenge@csirt.muni.cz. The code there should have:

  • a form that is conformant to AICA challenge, so that we can actually install it;
  • everything necessary to run - if it is using, e.g., some LLM it should also provide everything necessary to connect to it and use it for multiple runs;
  • a brief explanation of its working mechanisms.

After the deadline, we will evaluate the submitted agents. For an agent to be considered for the main prize, it has to:

  • be able to finish at least 3 scenarios of 5;
  • achieve a score comparable to the heuristic agent;
  • use some form of machine learning to drive the decision-making process.

The results will be announced on 31. 10. 2025.

Registration

If you register with us, we will email you the API key, and will be sending you further info on challenge progress. You can register anytime, but why wait...