Split auto-evo mutation generation into substeps

[RyuDanuer]

Brief Description of What This PR Does

This PR splits ModifyExistingSpecies mutation generation into resumable substeps, so the expensive mutation recursion no longer happens as one long auto-evo step. The mutation traversal now keeps explicit frame state and continues over many RunStep calls instead of recursively finishing a whole species in one call.

I also updated AutoEvoRun to refresh the total step estimate while steps are running, as ModifyExistingSpecies.TotalSteps now changes while its internal mutation work is consumed.

I manually tested this with the Auto-Evo Exploring Tool by running 5 worlds for 15 generations. The run completed, and the log had 0 matches for Single auto-evo step ModifyExistingSpecies. The measured worst ModifyExistingSpecies substep was 163.764 ms, under the 0.4s warning threshold.

AUTOMATED_AUTO_EVO_TOOL_TEST completed worlds=5 currentWorldGeneration=15 totalSpecies=209
ISSUE_5416_STEP_PROFILE step=ModifyExistingSpecies count=1828475 totalMs=889089.560 maxMs=163.764

I also ran:

dotnet build Thrive.sln -v minimal
# 0 warnings, 0 errors

dotnet test test\code_tests\ThriveTest.csproj --no-build --no-restore -v minimal
# Passed: 100, Failed: 0

Related Issues

Closes #5416

Progress Checklist

Note: before starting this checklist the PR should be marked as non-draft.

• PR author has checked that this PR works as intended and doesn't
  break existing features:
  https://wiki.revolutionarygamesstudio.com/wiki/Testing_Checklist
  (this is important as to not waste the time of Thrive team
  members reviewing this PR). This includes gameplay testing by the PR author.
• Initial code review passed (this and further items should not be checked by the PR author)
• Functionality is confirmed working by another person (see above checklist link)
• Final code review is passed and code conforms to the
  styleguide.

Before merging all CI jobs should finish on this PR without errors, if
there are automatically detected style issues they should be fixed by
the PR author. Merging must follow our
styleguide.

[RyuDanuer]

I manually reran the Auto-Evo Exploring Tool with 5 worlds x 15 generations and generated a fresh step timing flamegraph from that run:

The key part for this issue is the max single substep time, not the cumulative total. ModifyExistingSpecies is still where most of the auto-evo time goes overall, but it was split into a lot of small calls in this run:

AUTOMATED_AUTO_EVO_TOOL_TEST completed worlds=5 currentWorldGeneration=15 totalSpecies=209
ISSUE_5416_STEP_PROFILE step=ModifyExistingSpecies count=1828475 totalMs=889089.560 maxMs=163.764

I also checked the log for Single auto-evo step ModifyExistingSpecies and got 0 matches, so the manual run stayed under the 0.4s warning threshold for this step.

[Accidental-Explorer]

Besides the unsuccessful automated check:

Before I look in detail, did you make sure the in-game auto-evo results are equivalent to before this change?

[hhyyrylainen]

TotalSteps should never reduce. That doesn't make sense as the total steps should track the total number of steps that are done and are yet to come. So the code should be refactored to never reduce TotalSteps, it should only be allowed to increment.

[hhyyrylainen]

Why did you remove the volatile marking? As far as I can tell you did not remove threading from auto-evo so this is a totally wrong change to make.

[RyuDanuer]

Besides the unsuccessful automated check:

Before I look in detail, did you make sure the in-game auto-evo results are equivalent to before this change?

The split-step mutation generation preserves the old behavior more closely, especially by keeping the base mutant lifetime/order equivalent to the old recursive path. For validation, I ran a fixed-seed auto-evo comparison against the PR parent for 3 generations. The raw species IDs / generated names can differ, but after remapping those out, the actual auto-evo state matched: species traits, populations, and patch occupants were equivalent.

[hhyyrylainen]

I'm pretty sure this would break all other auto-evo steps than the one you modified.

Also rather than locking just for an int, compare exchange or other interlocked methods should be used.

Overall this still looks like a major LLM mess I'm not sure I will have the mental energy to review anytime soon.

[RyuDanuer]

I replaced the lock with an Interlocked.CompareExchange loop, so the total step estimate still only moves upward but doesn’t use a lock just for the int update. This keeps the volatile/threaded behavior intact while addressing the review point.

[hhyyrylainen]

You didn't address the fact that other auto-evo step types likely will not work correctly as their CompletedSteps assumes it approaches and reaches the total steps it reports and not like this.

I'm tired of trying to explain to you that bigger changes that touch the overall code architecture of Thrive need a lot more careful manual review than you are doing.

It might be the case that ModifyExistingSpecies.cs is converted reasonably and works without excessive temporary memory usage, but I'm tired of looking at AI PRs again and again when it would be faster for me to just redo the work from scratch.

For human programmers who want to get into Thrive I will try my hardest to provide constructive feedback on their PRs so that they might improve and in the future become a net contributor (or at least walk away from Thrive with some valuable experience on their programming career). But for your PRs I doubt they are going to get any better as I'm still needing to mention that you can't just let the AI run wild as it will not care about code uniformity or properly structuring code changes to fit in with existing code.

[hhyyrylainen]

You didn't address the fact that other auto-evo step types likely will not work correctly as their CompletedSteps assumes it approaches and reaches the total steps it reports and not like this.

I'm tired of trying to explain to you that bigger changes that touch the overall code architecture of Thrive need a lot more careful manual review than you are doing.

It might be the case that ModifyExistingSpecies.cs is converted reasonably and works without excessive temporary memory usage, but I'm tired of looking at AI PRs again and again when it would be faster for me to just redo the work from scratch.

For human programmers who want to get into Thrive I will try my hardest to provide constructive feedback on their PRs so that they might improve and in the future become a net contributor (or at least walk away from Thrive with some valuable experience on their programming career). But for your PRs I doubt they are going to get any better as I'm still needing to mention that you can't just let the AI run wild as it will not care about code uniformity or properly structuring code changes to fit in with existing code.

[hhyyrylainen]

If you want to be helpful to Thrive, I suggest you would look through our issue backlog and pick out small bugs that are much more likely for an AI to be able to solve on the first go rather than these bigger issues where the AI just clearly fails at refactoring existing code in a satisfactory manner. Small issues even if not high priority or planned to be a priority to work on, would improve Thrive and those would hopefully be ones were the AI agent can create nice and understandable at a glance PRs.