R&D Tactical bug hunts (suppression and alien skipping turn)

[Amineri]

So I'm looking at the function SwitchToAttack

 

It checks the obvious stuff like "UnitIsAliveAndWell" and that the unit is not currently in the process of moving, then performs:

 

m_kAbilityDM.UpdateTopShotOption();
if(m_kAbilityDM.HasShotAbility())
{
m_kAbilityDM.UseShotAbility(strReason);
ExecuteAbility();
}
else
{
if(m_kAbilityDM.UseLastResortAbility(strReason))
{
ExecuteAbility();
}
else
{
m_kAbilityDM.m_strAbilities.AddItem("Aborted move. Ended turn." @ strReason);
GotoState('EndOfTurn');
}
}
So the alien would have to fail its shot ability and also fail its "LastResort" ability before ending its turn with no action taken (assuming that it is starting the SwitchToAttack function, which it should be as long as the Shot_Standard ability is being scored positive, and it can't move)

[Amineri]

I've found the following restrictions and requirements related to the "basic" actions.

1 -- move

Restricted if :
Being Suppressed
Abort Move is true
Can Melee is true (I think this applies if a melee unit can currently melee attack a unit)

7 -- ShotStandard

Restricted if:
CanEngage is true

33 -- Overwatch

Restricted if:
Should Attack Civilians is true
CanEngage is true

Many many abilities are restricted if the CanEngage flag is set to true:

SetAbilityRestriction(7, 128); -- eAbility_ShotStandard
SetAbilityRestriction(48, 128); -- eAbility_PsiLance
SetAbilityRestriction(58, 128); -- eAbility_PsiDrain
SetAbilityRestriction(17, 128); -- eAbility_ShotSuppress
SetAbilityRestriction(33, 128); -- eAbility_Overwatch
SetAbilityRestriction(22, 128); -- eAbility_FragGrenade
SetAbilityRestriction(24, 128); -- eAbility_AlienGrenade
SetAbilityRestriction(51, 128); -- eAbility_GreaterMindMerge
SetAbilityRestriction(47, 128); -- eAbility_MindMerge
SetAbilityRestriction(75, 128); -- eAbility_MindControl
SetAbilityRestriction(52, 128); -- eAbility_PsiControl
SetAbilityRestriction(53, 128); -- eAbility_PsiPanic
SetAbilityRestriction(56, 128); -- eAbility_ReanimateAlly
SetAbilityRestriction(57, 128); -- eAbility_ReanimateEnemy

If the CanEngage flag is not being set to false if the unit fails to move, the all of these abilities are restricted and hence unavailable.

I'm looking for ways that the flag is set to false now.


---------------

on a side note, as a point of interest

38 -- eAbility_TakeCover

Requires:
(Suppressed or overwatched) AND No flankers
or
No Friends AND More Enemies than friends

These are the conditions under which an alien can possibly "hunker down"

 

------------------------------

 

Edit: my mistake... the abilities are restricted if the CanEngage is FALSE. This is how the AI limits the number of active units based on difficulty for easy and normal difficulties. With MaxActiveAIUnits[eDifficulty] == -1, CanEngage will always return true, so the above ability list will always pass the restrictions test.

 

Back to looking ...

Edited May 7, 2013 by Amineri

[Amineri]

It appears I was mistaken in my earlier belief that the AI selects the highest priority action. It instead does performed a weighted random roll across all actions with positive priority.

 

For example, if Action1.priority = 10 and Action2.priority = 20, then 1/3 of the time the AI will select Action1 and 2/3 of the time it will selects Action2.

 

This is generalized across any number of actions, and shows quite clearly why actions with negative priority are not included. This function should still work correctly even if there is one action with priority 1 left in the array of allowed actions.

 

However, it may explain why you might see somewhat randomized or inconsistent actions taking place.

 

Unfortunately, I don't feel much close to figuring out why units are skipping their turn...

[johnnylump]

FWIW, Changing the move score to a minimum of 255 seemed to have some effect in general -- aliens frequently ran into the open. I couldn't reproduce the freeze, but it was one battle (including flanked-overwatched enemies) and needs more testing. (Not a solution anyway, but a data point in determining whether it's something in the scoring, or something elsewhere that's causing the freezes). I'd note again that the aliens froze when overwatch was scored at a minimum of 1, too.

 

Worth noting the freeze behavior happens when the alien is suppressed.

 

In SwitchToAttack:

 

The call to HasShotAbility() is the only call to that function in the entire upk.

The call in HasShotAbility() to HasBackupShotCapability () in the only call to that function in the entire upk. (There are two functions with that name, one in XGAIBehavior, which always returns false, and one in XGAIBehavior_Psi which can return true under a couple of conditions.

The call in SwitchToAttack() to UseShotAbility () is the only such call in the entire upk.

 

I'm pretty sure ExecuteAbility() isn't decompiling correctly, but presumably it ends up at GotoState 'AttackState' in most cases.

Edited May 7, 2013 by johnnylump

[Amineri]

I think I've worked out the top-level AI control flow.

 

1) The unit's behavior goes from state 'inactive' to 'active'

 

2) state 'active' does two things:

a) InitTurn (creates, filters and prioritizes all of the abilities -- amongst other things)

b) GotoState('ExecutingAI')

 

3) state 'ExecutingAI' does the following (after removing debug/dev-console/glam-cam code)

 

 

state "ExecutingAI"

if(m_kUnit.IsAliveAndWell())

{

if(m_kAbilityDM.m_nAbilities == 0)

{

GotoState('EndOfTurn');

}

else

{

if(m_kUnit.IsAffectedByAbility(55))

{

GotoState('Berserk');

}

else

{

m_kAbilityDM.SelectAbility();

if(m_kAbilityDM.IsValidAbility())

{

ExecuteAbility();

}

// End:0x831

else

{

GotoState('EndOfTurn');

}

}

}

}

DoStateTransition();

stop;

The actual selection and execution of the abilities is done via the m_kAIAbilityDM.SelectAbility and ExecuteAbility calls

 

4) At the end of state 'ExecutingAI', DoStateTransition is called

 

5) DoStateTransition checks for the end-of-turn conditions. If not the unit's end-of-turn, then it calls RebuildAbilityList to refilter/prioritize the abilities based on the new conditions and goes to state 'ExecutingAI'

 

simulated function DoStateTransition()
{
// End:0x68
if((!m_kUnit.IsAliveAndWell() || m_kUnit.CheckForEndTurn()) || m_bAbortMove)
{
GotoState('EndOfTurn');
}
// End:0xA0
else
{
m_kAbilityDM.RebuildAbilityList(false);
InitExecuteAbility();
GotoState('ExecutingAI');
}
//return;
}

 

 

-------------------------------------------

 

Unfortunately, this hasn't given me any additional insights into why the aliens are turn-skipping. There is code that will "time out" the AI if it is taking too long to generate a move. It could be that some flaw somewhere is causing the AI to enter an 'infinite loop', causing the timer to time and force the unit to end its turn.

 

The AddAbilityOption function allows for an optional boolean bForcePriority (which is presumably designed to force near 100% activation of this ability). It works by adding 1,000 to the ability's priority before setting it. This should give an idea of priority scaling... the designers considered 1,000 to be "close to infinity"

 

------------------------------------------

 

It is possible for aliens to utilize "Dashing" moves.

 

In XGAIBehavior.GetBestFlankingPointAgainst, the code begins with the call:

m_kUnit.SetDashing(true)

 

This allows Dashing moves as valid destinations, giving the unit a wider variety of valid destinations to pick from while trying to flank a unit.

 

m_kUnit.SetDashing(false); is set during XGAIBehavior.InitTurn, and SetDashing(true) is not set during any other calls in XGAIBehavior (not even in state 'Flee'), so flanking is the only condition under which aliens are currently allowed to dash.

 

It might be possible to add a SetDashing(true) call if the unit is flanked, allowing it to try and dash for a safe location, which would increase the chances of finding a non-flanked position as well as reducing the shot odds. This would likely require a reworking of the move priority factor 'number of overwatching units'

 

-----------------------------------------

 

Unrelated to the "skipping turn" problem (but still in the R&D tactical bug hunt), the XGAIBehavior.WarpTo() function is what appears to implement the aliens ability to skip the normal pathfinding/moving sequences.

 

By identifying where it is used, and forcing only the alternatives, it may be possible to prevent the AI from EVER teleporting aliens, forcing them to always use the pathfinding/movement code. This would firmly squash the teleport bug, albeit at the cost of longer alien turns.

[Amineri]

I went ahead and implemented Amineri's suggestion. Turns out there was only space for a single conditional check, so I followed my heart and left it shooting. The game loads up with these changes, and battles run, but I have not tested them enough yet to see if the alien freeze is eliminated.

 

Here's the code:

function int AI_GetScore(XGAbility kAbility, out string strAbilities)
{
    local float fOS, fDS, fIS;
    local int iOS, Ids, iIS;

    // End:0x11
    if(kAbility == none)
    {
        return 0;
    }
    fOS = AI_OffenseScore(kAbility);
    fDS = AI_DefenseScore(kAbility);
    fIS = AI_IntangiblesScore(kAbility, fOS, fDS);
    iOS = int(float(100) * fOS);
    Ids = int(float(100) * fDS);
    iIS = int(float(100) * fIS);
    // End:0x12F
    if(kAbility.iType == 8)
    {
        return Max(int(float((iOS + Ids) + iIS) * (AI_GetScoreModifier(kAbility))), 1);
    }
    // End:0x168
    else
    {
        return int(float((iOS + Ids) + iIS) * (AI_GetScoreModifier(kAbility)));
    }
    //return ReturnValue;    
}

 

 

And here are the hex changes:

 

XcomGame
3a 01 00 00 cc 28 00 00 6b 01 00 00 07 01 00 00
to
3a 01 00 00 cc 28 00 00 73 01 00 00 07 01 00 00

AND

0f 48 dd 88 00 00 70 a8 70 a8 70 a8 1b b5 02 00 00 00 00 
00 00 00 de 88 00 00 16 1f 4f 66 66 3d 00 16 38 53 00 d8 
88 00 00 16 1f 44 65 66 3d 00 16 38 53 00 d7 88 00 00 16 
1f 49 6e 74 3d 00 16 38 53 00 d6 88 00 00 16

to

07 2f 01 9a 19 00 38 89 00 00 09 00 e2 7b 00 00 00 01 e2 
7b 00 00 2c 08 16 04 fa 38 44 ab 38 3f 92 92 00 d8 88 00 
00 00 d7 88 00 00 16 00 d6 88 00 00 16 1b b7 02 00 00 00 
00 00 00 00 de 88 00 00 16 16 26 16 06 68 01

 

(It was an exact fit -- not a single 0b needed) Please give it a try and report if the issue is fixed!

 

Your hex code above references the wrong version of the local variable kAbility. kAbility is a LOCAL variable to the function, not a class variable. It looks like you got the reference to kAbility.iType from IsABetterShotOption, which has the a local variable of the same name (kAbility).

 

However, the correct reference to kAbility in AI_GetScore is 00 DE 88 00 00.

 

This should make the construction for kAbility.iType (in AI_GetScore):

 

19 00 DE 88 00 00 09 00 E2 7B 00 00 00 01 E2 7B 00 00

 

instead of

 

19 00 38 89 00 00 09 00 E2 7B 00 00 00 01 E2 7B 00 00

 

With this incorrect construction (I'm surprised the game didn't crash), the wrong ability would have been tested, which would have resulted in incorrect results.

 

I'll be testing out the game with the updated kAbility.iType reference and will hopefully have good news :D

[johnnylump]

(Hangs head in shame)

 

(wonders if we can borrow local variables from other functions in the same class)

Edited May 7, 2013 by johnnylump

[Amineri]

I used the construction: (which leaves lots of extra bytes)

 

function int AI_GetScore(XGAbility kAbility, out string strAbilities)
{
local float fOS, fDS, fIS;
local int iOS, Ids, iIS;

// End:0x11
if(kAbility == none)
{
return 0;
}
fOS = AI_OffenseScore(kAbility);
fDS = AI_DefenseScore(kAbility);
fIS = AI_IntangiblesScore(kAbility, fOS, fDS);
iOS = int(float(100) * fOS);
Ids = int(float(100) * fDS);
iIS = int(float(100) * fIS);
return Max(int(float((iOS + Ids) + iIS) * (AI_GetScoreModifier(kAbility))), ((kAbility.iType == 7) ? 1 : 0));
}

 

 

Hex replacement is:

 

 

Hex (original)

F3 88 00 00 50 55 00 00 00 00 00 00 D5 88 00 00 00 00 00 00 00 00 00 00 DE 88 00 00 00 00 00 00 3A 01 00 00 CC 28 00 00 6B 01 00 00 07 01 00 00 07 11 00 72 00 DE 88 00 00 2A 16 04 25 0F 00 DB 88 00 00 1B BC 02 00 00 00 00 00 00 00 DE 88 00 00 16 0F 00 DA 88 00 00 1B B3 02 00 00 00 00 00 00 00 DE 88 00 00 16 0F 00 D9 88 00 00 1B B9 02 00 00 00 00 00 00 00 DE 88 00 00 00 DB 88 00 00 00 DA 88 00 00 16 0F 00 D8 88 00 00 38 44 AB 38 3F 2C 64 00 DB 88 00 00 16 0F 00 D7 88 00 00 38 44 AB 38 3F 2C 64 00 DA 88 00 00 16 0F 00 D6 88 00 00 38 44 AB 38 3F 2C 64 00 D9 88 00 00 16 0F 48 DD 88 00 00 70 A8 70 A8 70 A8 1B B5 02 00 00 00 00 00 00 00 DE 88 00 00 16 1F 4F 66 66 3D 00 16 38 53 00 D8 88 00 00 16 1F 44 65 66 3D 00 16 38 53 00 D7 88 00 00 16 1F 49 6E 74 3D 00 16 38 53 00 D6 88 00 00 16 04 38 44 AB 38 3F 92 92 00 D8 88 00 00 00 D7 88 00 00 16 00 D6 88 00 00 16 1B B7 02 00 00 00 00 00 00 00 DE 88 00 00 16 16 04 3A DC 88 00 00 53

Hex (new) (virtual size 0x15F)

F3 88 00 00 50 55 00 00 00 00 00 00 D5 88 00 00 00 00 00 00 00 00 00 00 DE 88 00 00 00 00 00 00 3A 01 00 00 CC 28 00 00 5F 01 00 00 07 01 00 00 07 11 00 72 00 DE 88 00 00 2A 16 04 25 0F 00 DB 88 00 00 1B BC 02 00 00 00 00 00 00 00 DE 88 00 00 16 0F 00 DA 88 00 00 1B B3 02 00 00 00 00 00 00 00 DE 88 00 00 16 0F 00 D9 88 00 00 1B B9 02 00 00 00 00 00 00 00 DE 88 00 00 00 DB 88 00 00 00 DA 88 00 00 16 0F 00 D8 88 00 00 38 44 AB 38 3F 2C 64 00 DB 88 00 00 16 0F 00 D7 88 00 00 38 44 AB 38 3F 2C 64 00 DA 88 00 00 16 0F 00 D6 88 00 00 38 44 AB 38 3F 2C 64 00 D9 88 00 00 16 04 FA 38 44 AB 38 3F 92 92 00 D8 88 00 00 00 D7 88 00 00 16 00 D6 88 00 00 16 1B B7 02 00 00 00 00 00 00 00 DE 88 00 00 16 16 45 9A 19 00 DE 88 00 00 09 00 E2 7B 00 00 00 01 E2 7B 00 00 2C 07 16 02 00 2C 01 02 00 2C 00 16 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 53

 

I first tested it with a minimum shot priority of 1, but was still able to induce a paralyzed sectoid on the first turn. However, I left space for larger values, so am going to try larger default priorities for shotstandard and see what happens.

[Amineri]

Tested it up to a value of priority 255 -- aliens still skip their turns if flanked and overwatched (even if mind-melded). They are a lot more likely to shoot now, even at units in high-cover (lots of cover destruction going on now). It appears that clamping shot priority to no less than 255 does have an effect on the AI, but is not sufficient to prevent a unit from skipping its turn.

 

I'm going to play around with some other values next.

[Amineri]

Am slowly tracking it down, I think.

 

There are two separate and independent scoring processes that go on:

 

1) XGAIAbilityDM.AddAbilityOption

 

This scores each ability, based on a variety of factors such as alien's position, what enemies it can see, whether it is flanked, suppressed, to-hit chances, etc. If the priority score of the ability is positive, it goes onto the list of possible abilities. Every positively scored ability has a chance to be executed -- they are randomly selected, but weighted based on relative priority.

 

2) XGAIPlayer.CalculateCoverScore / CalculateDestinationScore

 

This scores each location on its desirability. This is done independently of any particular alien or ability. It takes into account how many enemies can be seen, distance to enemies, whether the position is flanked, how many overwatching enemies can see the positions, etc.

 

The AI calculates the valid destinations at the beginning of its turn, and also recalculates them each time an alien unit moves or takes an action, as some of the values include distance to friends. Also, XCOM units could be killed or use their overwatch shots which could alter the scores of various locations.

 

-------------------------------------------------------------------------

 

A unit decides whether the move is better than another option BEFORE it looks at the list of possible destinations.

 

For a sectoid being flanked and overwatched, the default priority for shooting is negative ... it is prohibited from being placed on the list.

 

For our particular situation (sectoid flanked and overwatched, and all visible enemies in high cover -- all shots at 35% or less)

 

MoveOffense score (calculated in XGAIBehavior.GetMoveOffenseScore)

The function does not check if the target is flanked ... only if it is in cover

Since it IS in cover, it returns priority = 100 - HitChance = 65 (for vanilla) or 75 (for Long War)

 

MoveDefence score (calculated in XGAIBehavior.GetMoveDefenceScore)

This function does check if the unit is in cover and has flankers.

In that case it returns priority = 100

 

So total move priority for the unit is 165 (175 for Long War). This is a pretty high priority. If the Standard Shot priority were 255, there is still a 40% chance that the unit would attempt to move instead of shooting.

 

With the default Standard Shot priority at 1, there is a tiny < 1% chance that the unit will shoot instead of attempting to move.

 

---------------------

 

So what happens when it tries to move?

 

The order of checking move options is:

 

1) If Can't Engage, then 'Roam' (this only applies to Easy/Normal difficulties when the number of engaging aliens is limited)

2) If HasValidManeuver, then "MoveManeuver'

3) If HasCustomMoveOption, then EnterCustomMoveState (this applies to Floaters, Drones, Ethereals and Sectoid Commanders)

4) If Not seen, then 'Roam'

5) If ShouldIgnoreCover, then 'IgnoreCoverMove' (for melee units and tanks that ignore cover)

6) If HasScoredTacticalOptions, then 'TacticalMove'

7) If ShouldFlee, then 'Flee'

8) If ShouldFlank, then 'Flank'

9) If ShouldEngage, then 'Engage'

 

In particular the ShouldFlee, then 'Flee' looks likely for a flanked target.

 

Interestingly, sectopods have this overwritten so that they NEVER flee.

Terrorists on terror missions never flee

Units that should keep hidden always flee (mind merge originator units)

Units that can see enemies and use cover should flee if they ShouldFindCover or if they are InCover and AreExposed

 

This means that the flanked and overwatched alien is going to enter the 'flee' state, which is an extension of the 'MoveState' state. The 'flee' state is different in that DecideNextDestination function only looks for places to flee to (not good places to shoot from, for example).

 

if(!FindRunawayPosition(vDestination, kEnemy, VDir, DistSq, strFail))

{

SwitchToAttack(strFail);

}

return vDestination;

 

However, dashing moves are not allowed even for fleeing (which I think should be remedied). If the unit cannot find a better destination with a single move, the it defaults to the failsafe SwitchToAttack function. This function is really meant to catch "soft" errors, and does not take into account the ability priorities that were set earlier.

 

It sets m_AbortMove = true, which will force the unit to end its turn the next time DoStateTransition is called. It's only remaining options left are in the SwitchToAttack function. This consists of the code snippet:

m_kAbilityDM.UpdateTopShotOption();

// End:0x112

if(m_kAbilityDM.HasShotAbility())

{

m_kAbilityDM.UseShotAbility(strReason);

ExecuteAbility();

}

 

 

This code doesn't use any of the priorities, and instead just tries to shoot at something as a sort of failsafe. If this fails then the unit will end its turn having performed no actions.

 

------------------------------

 

The solution is two-fold.

 

1) Allow fleeing units to dash -- this will open up the range of possible locations the unit can move to

 

2) Find what is causing the shot ability to fail and fix it so that if even with dashing escape isn't viable the unit will shoot as a backup.

 

I'll be working on it :)

Edited May 7, 2013 by Amineri