Submit flags for an instance that is no longer running

[vincentscode]

When playing a dynamic challenge, the user may wish to submit the acquired flag, after the instance has already expired.
This is especially true, if a solution can also be found locally, without launching an instance and the instance is only provided to make it easier to get started.

Currently the user is met with this error message:

ctfd-chall-manager/models.py

Lines 450 to 465 in c70e28b

	# If the instance no longer exists
	if data["since"] is None:
	logger.debug(
	"instance for source_id %s and challenge_id %s no longer exists",
	source_id,
	challenge.id,
	)
	logger.info(
	"invalid submission due to expired instance for challenge %s source %s",
	challenge.id,
	source_id,
	)
	return ChallengeResponse(
	status="incorrect",
	message="Expired (the instance must be running to submit)",
	)

If there is no technical reason this is impossible, I would like to be able to allow such submissions.

[NicoFgrx]

Hi,

There are indeed technical and organizational reasons that justify this choice.
We listed some of them here: https://ctfer.io/docs/ctfd-chall-manager/learn-more/flag/
If this doesn’t fully answer your questions, we may need to update the documentation?

More specifically, on our side we use the flag-variation engine on almost all of our instances:

• https://ctfer.io/docs/chall-manager/design/software-development-kit/#flag-variation-engine
• https://ctfer.io/docs/chall-manager/challmaker-guides/flag-variation-engine/

In this context, each instance has its own unique flag(s). We do not store these flags at the plugin / CTFd level.
So the instances must be running in order to verify whether a submitted flag is valid.

In your case, I imagine you fall under this situation: https://ctfer.io/docs/ctfd-chall-manager/learn-more/flag/#why-must-the-instance-be-on-to-submit ?

[vincentscode]

Hi, I read all the docs (even though they are unfortunately very out of date or sparse in places - I will create follow-up issues if I find the time) and understand why you chose to require the instance to be running.

I also understand that the flag information is not available at the CTFd plugin.
What I was wondering is if the information is available at the chall-manager deployment - I would assume that to be the case even without the instance actually running.
That would make it possible to submit flags without an instance running (if it ever ran before perhaps).

You did choose not to enable this by default to prevent hoarding, but as described in my previous comment the annoyance for some challenges outweighs the risk of flag hoarding in my use-case. Besides that you can still hoard solution-scripts instead of flags, which has the same effect on the competition.

[NicoFgrx]

Indeed, if you have automated the resolution, we cannot combat FlagHoarding, but we assumed that it would require additional effort on the part of the players, and therefore it would slow down their progress in the competition.

That's also why we combine having to leave the instance running AND the mana cost of the challenge AND the dynamic flag: if you want to do FlagHoarding, okay, but you won't be able to have other instances open at the same time because you'll be limited by mana, or you'll have to make the effort to automate your solve because the flag won't be the same for your new instance.

In your case, if I understand correctly the discussion at #226, you don't use mana.

Apart from a case of FlagHoarding, I don't see how you can solve a challenge that requires an instance that isn't there anymore.

[vincentscode]

I have explained why it would be nice to be able to allow this here:

When playing a dynamic challenge, the user may wish to submit the acquired flag, after the instance has already expired.
This is especially true, if a solution can also be found locally, without launching an instance and the instance is only provided to make it easier to get started.

[NicoFgrx]

if a solution can also be found locally, without launching an instance

I don't understand how this can be possible... if you can find the solution locally and you don't need any instances why you don't just do a CTFd's dynamic type challenge ? (not a dynamic_iac).

I think is a missunderstanding from my part, maybe a scenario with concret example can help ?

[vincentscode]

Just to give the user options. Usually a rev challenge can be solved fully locally. Giving the user a hosted variant to play with makes it more accessible however.

[NicoFgrx]

Okay, in that case, I think it should be a configurable element at the challenge level with a boolean like destroy_on_flag while maintaining the current behavior. Let's say require_instance_on_to_submit with true as the default value?

We will therefore need to add warnings about certain incompatibilities to the documentation:

• we will not be able to use a flag generated by chall-manager (generated flags cannot be stored in CTFd database cf docs)
• we will not be able to guarantee no FlagHoarding on these challenges (like those without the plugin)

In the short term, I won't have much time to do the implementation, but a PR would be welcome :)

[vincentscode]

we will not be able to use a flag generated by chall-manager (generated flags cannot be stored in CTFd database cf docs)

Why would it need to be in the CTFd database? We can still use the chall-manager deployment to validate the flag.

[NicoFgrx]

In the plugin, we have two sources of flags for a given challenge:

1. The flags configured in CTFd, common to all teams and players
2. The flags generated by Chall-Manager, unique to the instance you deployed

If we store the flags generated by Chall-Manager in the CTFd database, those instance-specific flags become usable by anyone, which defeats the purpose of preventing flag sharing. This is why we absolutely need the running instance to perform flag verification.

=> If your challenge does not define any instance-specific flag, then you simply use the standard CTFd system.
=> If you can retrieve the flag locally and do not want to require the instance to be running, then you should use the standard CTFd flag system because we can't verify a instance specific flag that does not exist anymore.

[D-VR]

those instance-specific flags become usable by anyone

This may not directly prevent flag sharing, but would make it possible to keep track of which flags were generated for which team.
So instead of pro-actively preventing flag sharing, you keep track of which teams shared which flags to which other teams.
While allowing flag submission even if the instance is offline. (you could also still prevent instance-specific flags from becoming usable by anyone checking the team it was generated for during submission)

This would allow you to automatically generate post CTF cheating reports similiar to what GPN23 did: https://kitctf.de/gpnctf-23/gpn-ctf-flagshare

Of course this would be a big change to how it's currently implemented, but something to keep in mind if you want to revisit this in the future.

[NicoFgrx]

Hi,
I think we could consider a mode where we can enable or disable the submission of expired flags:

1. "Detection" mode: Accepts flag submissions from other teams, but alerts administrators of potential cheating.
2. "Protection" mode: Blocks flag submissions from other teams and alerts administrators of potential cheating.

In both cases, platform administrators are notified and can trace which flags were submitted by whom and to whom they belong, etc.

However, such a feature would require significant refactoring (e.g stores flags on CTFd database), which cannot be done quickly. In any case, we need to discuss this again with @pandatix, as we had planned to create a dedicated anti-cheat component in our roadmap, but it is not currently a priority.