Creating an A.I that communicates with other squadmates

Remco Spil

Table of contents

1. Introduction
2. Pathfinding
   1. Choices
   2. Solution
3. Flanking
   1. Solutions
   2. What went well
   3. What went wrong
4. Cover seeking
5. Flushing the player out
6. Swarming
7. Final result
8. Future work
9. Sources
10. Foto

1. Introduction

In a lot of games, we come across A.I look at any multiplayer game that has bots like battlefield, Cod, and Division. The bots find a player seeking cover and engage with the player. Some bots have more advanced A.I that allows them to flank the player or push the player to fight them when they are low. A great example and a lot of inspiration come from the game F.E.A.R which is known for its amazing A.I that communicate well with each other and work together to fight the player. In the future, I wanted to make my own game with my own A.I that could do these kinds of things.

So for this project I wanted to create an A.I that communicates with other A.I to perform a few tasks. These tasks are flanking the player from 2 different locations. Hiding behind cover when shot. Trying to flush out the player with a grenade if hiding behind cover for too long and swarming the player with numbers if the player is either low or there are a lot of enemies to swarm with.

Inspiration

for this project, I took inspiration from a video on youtube explaining the A.I of fear. In the video, you can see enemies creating a cover, flanking the player, flushing the player out with grenades, and fighting with groups.

2. Pathfinding

Before we can make the flanking and swarming of the A.I we’ll need to decide on pathfinding. With pathfinding, we can make the A.I automatically find the shortest path toward the player while also avoiding all the obstacles around them.

Choices:

• Navmesh
• NavMesh Plus
• A* Dijkstra

Navmesh

Navmesh is a mesh that defines walkable areas. These areas can have higher or lower costs making it unfavorable or favorable to walk over.

Navmesh uses agents that have a specific width to avoid obstacles or other agents. You can make specific agents have a higher cost for certain areas while other agents do not have those higher costs. An issue that might happen is that while there is obstacle avoidance it has no clue how to walk around another agent and will just push the agent aside

NavMesh Plus

NavMesh Plus is the same as Navmesh but it works in 2D by using the Unity tile renderer. This is not supported by Unity but works just as well as the 3D version.

A* pathfinding

A* pathfinding is a quick pathfinding algorithm that uses graphs to find the shortest path. It checks which unchecked neighbor has the smallest distance to the end goal and checks if they can traverse over this tile. A* is not built into Unity and you have to either make it yourself or add a package from a website.

Solution

For the project, I didn’t want to work in 3D unity as I am more known for 2D games and top-down games. As I will only simulate the A.I behavior I didn’t find it necessary to make in 3D. I have therefore chosen to work with Unity’s Navmesh pathfinding and then specifically the 2D version Navmesh plus. The reason for not choosing A* is because there is less documentation available on the internet and Navmesh has good documentation on how to manipulate the path

3. Flanking

What exactly is flanking. According to (Wikipedia contributors, 2022b)

The flanking maneuver is a basic military tactic with several variations. Flanking an enemy means attacking from one or more sides, at an angle to the enemy’s direction of engagement.

During this project, I spend most of my time working on adding a flanking mechanic. As the flanking mechanic is difficult to implement and even big game studios can’t seem to hit the nail every time. I started researching different methods and rules to simulate the flanking A.I. 

The games that the flanking should work on are fps games. So I wouldn’t be working with massive amounts of enemies but squads of a few normal enemies and a few enemies that would have the opportunity to flank the player. From the research that I did, I came up with 3 solutions that could work in an fps game.

Solutions

• Mark the path one A.I take it with a layer making the other A.i choose it less likely
• Avoid the path that the player is looking at.
• Draw nodes around the area the player is in and let their path through the nodes

What went well

Draw nodes around the player to flank the area the player is in not the player itself

I got the solution from a post on Reddit explaining in images how to make an A.I flank

Drawing nodes

At the start of the game, the player will draw a set number of lines around them. The more lines you make the better the precision is to find a suitable flank spot. The lines go till a certain distance and place a node at the end. This creates an area where the player is in which the A.I will try and flank. If the player reaches the edges of the area they will redraw all the nodes on the player’s location again. I have done this because it saves a lot of resources to not draw and check the availability of a flank node and because flanking an area that the player is in is enough to actually flank.

Checking flank availability

After we draw the nodes we check for every node in the list to see if there is a wall between it and the player. If there is no wall between the player and the wall then we mark it green and add it to a list of suitable flank positions. We mark the node yellow if there is a wall between the node and the player. We then give every flanker access to this list to choose a suitable destination.

Problem 1: Node stuck inside the wall

Solution:

For every note, there will be a check if it’s inside a wall right now. If they are in a wall they will move the node 1 extra step in its previously stated direction. Then there is a ray going towards the player that will collide once with the wall. This position is saved and then we move the node 1 extra step further. If the collision position is the same then the node is outside of the wall.

    public void CheckWallCollision(Vector3 direction, Vector3 point, int number)
    {
        Vector3 startLocation = point + direction;

        for (int i = 0; i < 100; i++)
        {
            RaycastHit2D hit = Physics2D.Linecast(startLocation, homePoint, layer);

            if (hit)
            {
                float distance = Vector3.Distance(hit.point, homePoint);

                if (distanceToPlayer == 0)
                {
                    distanceToPlayer = distance;
                }
                else if (distanceToPlayer == distance)
                {
                    _FlankNodes[number].transform.position = hit.point + (Vector2)direction;
                    Vector3 wantedposition = homePoint + (_FlankDirections[number] * flankNodeDistance);

                    float distance1 = Vector3.Distance(_FlankNodes[number].transform.position, homePoint);
                    float distance2 = Vector3.Distance(wantedposition, homePoint);

                    if (distance1 < distance2)
                    {
                        _FlankNodes[number].transform.position = wantedposition;
                    }
                    return;
                }
                else
                {
                    distanceToPlayer = distance;
                }
                startLocation = hit.point + (Vector2)direction;
            }
        }

        _FlankNodes[number].transform.position = homePoint + (_FlankDirections[number] * flankNodeDistance);
    }

Problem 2: Node is too close to the player for a working flank.

Solution:

Nodes that hit the outer wall will be placed at the edge of the wall. Some of the nodes get too close to the player to be a reasonable flank.

To solve this we make it that the nodes need to be a set destination from the player to be seen as a suitable flank.

Giving the path

When we are done finding out what flank nodes are available then we will send this information to the flankers. They will pick out a random available node. From there they will find the closest node and make the path from that closest node to the desired flank node.

Feedback 1: Add priority to the nodes that the A.I choose

Instead of letting the other A.I’s look for a random note to flank from. The flanker will choose a random node to flank from. After that we let all the other A.I’s choose a node depending on a list of priority nodes. The list is calculated by taking the node that the first flanker will go to and calculating the angle between that node the player and the current node we are examining. It then takes the node that scored the highest as a suitable flank node making sure that the player gets shot from the biggest difference is angles.

Waiting for other flankers

The last part is making the flanker wait for the other flanker. We do this to make sure that the player can’t deal with 1 and then the other. For this, I placed a few spots that are covered from the player’s vision. I choose to do this manually as my focus for this project wasn’t on finding a dynamic cover system. When the flankers reach a specific range of the player they will scout around for the cover that is the closest to the player and to the path that they are taking to the player. Once the A.I reaches the cover it will go into a new state where they will check if the other flankers have reached their cover. Once both flankers have reached the cover they will continue their path till the player is in vision.

What didn’t go well

Avoid the path that the other A.I am taking

The first thing I tried to do was make the flanker A.I avoid the path that the normal A.I was taking. The first step I did was to create a new tilemap and called this weighted path. I then showed the corners that the A.I would move.

The black is the corners where the direction of the A.I change. The blue is a path filled in based on the direction that the A.I is going. I would make this path with a weight of 100 so the next A.I would choose a different path.

            for (int i = 0; i < agent.path.corners.Length - 1; i++)
            {
                calculatePathPositions(agent.path.corners[i], agent.path.corners[i + 1]);
            }

public void calculatePathPositions(Vector3 start, Vector3 end)
        {
            //gets the start and end position on the tilemap
            Vector3Int startPos = tiles.WorldToCell(start);
            Vector3Int endpos = tiles.WorldToCell(end);

            //meassure how many steps it has to take to reach the next corner of the pathfinding
            float step = Vector3Int.Distance(startPos, endpos);
            steps = Mathf.Abs(startPos.x - endpos.x) + Mathf.Abs(startPos.y - endpos.y) -1;

            Vector3 dir = endpos - startPos;
            dir.Normalize();   

            //Makes the tilemap tile edible
            tiles.SetTileFlags(tiles.WorldToCell(newpos), TileFlags.None);
            tiles.SetColor(tiles.WorldToCell(newpos), Color.black);

            //Goes through a loop that sets the tile on the tilemap and add a tile above and under or right and left to make a thicker path
            for (int i = 0; i < step; i++)
            {
                newpos = newpos + dir;


                Vector3Int pos = tiles.WorldToCell(newpos);
                tiles.SetTile(pos, tile);
                if ((dir.x > dir.y && dir.x > 0))
                {
                    tiles.SetTile(pos + new Vector3Int(1,0,0), tile);              
                    tiles.SetTile(pos + new Vector3Int(-1, 0, 0), tile);
                } 

                if(dir.y > dir.x && dir.y > 0)
                {
                    tiles.SetTile(pos + new Vector3Int(0, 1, 0), tile);
                    tiles.SetTile(pos + new Vector3Int(0, -1, 0), tile);
                }
            }

        }

The problem I started to run into was that it’s unknown how thick the openings in the wall are and a flanker A.I could slip through a few gaps making the flank completely fail. Being stuck with this problem I went on a search to find a different solution which I found so I gave up on this solution.

Avoid the path the player is looking at:

I tried going this approach after trying to make the ai avoid the other ai path. I quickly started walking into problems with how the navmesh worked as it was hard to draw a tilemap for what the player saw. Another problem I walked into was that if the player looked at the only other remaining path the A.I just wouldn’t flank. I decided to not work further on this solution as it didn’t seem likely to bring any results

4. Cover seeking

For the cover-seeking part, I wanted to make a small decision to either have the flankers hide before they attack the player. Have the normal enemies keep the players from escaping their cover and running away.

On the map, I placed a set amount of covers. For each cover, I do a check if they are seen by the player and if they are in the range of the players. When the game starts I first give the flankers their path to flank with and the node they will be hiding behind. Every node that has been selected will be marked as used and cannot be picked by a different flanker or normal A.I. Once the flankers have chosen their covers I make the normal A.I choose their cover spot with a simple distance check to see what cover is not seen by the player and is closest to the player.

    public void UpdateNodes()
    {
        foreach (GameObject _cover in covers)
        {
            RaycastHit2D ray = Physics2D.Linecast(_cover.transform.position, player.position, layer);

            if (!ray)
            {
                if (!found.Contains(_cover))
                {
                    found.Add(_cover);
                    hideable.Remove(_cover);
                }
            }

            if (ray)
            {
                if (!hideable.Contains(_cover))
                {
                    found.Remove(_cover);
                    hideable.Add(_cover);
                    NodesInRange();
                }
            }
        }
    }

Hiding when getting shot by the player

When 1 of the 2 flankers would get shot they would hide behind the closest wall away from the player’s sight just like how real people would hide when they get shot at.

5. Flushing the player out

When the player sits in the same spot for too long it will be bad for the A.I to try and fight it. it would be better if you try and flush the player out. I have achieved this by checking if the player moves away from an invisible barrier around the player. If the player stays too long in the same barrier a grenade sign will spawn warning the player that they have to move. This can only be done once the normal A.I have found cover and the player has been standing in the same area for a long time.

6. Swarming

What is swarming?

According to (Wikipedia contributors, 2022a)

swarming is a battlefield tactic designed to maximize target saturation, and thereby overwhelm or saturate the defenses of the principal target or objective.

The last thing that I started working on is swarming. In the project, I let swarming be decided by how many times the player would have been hit and dependent on the number of enemies there are.
When the flankers are in the range of the player they let out a simple raycast that checks if they hit the player. I stop the raycast for 2 seconds after they hit a target. Once the player is hit enough the 2 remaining normal enemies will come out of their cover to start attacking the player

7. final Result

Overall I am pretty happy with the results. The flanking was a success where they choose the most optimal area to flank from. It is running completely fine. Also the swarming and the flushing has worked well. The cover seeking could have some upgrades so that I don’t have to place the nodes myself. The project itself got off on a rough start but after finding what seemed like a good solution I started to make a lot of progress in the flanking.

8. Future works

For the future of this project, I want the A.I to move better when the player comes out of their hiding spot and starts running towards other areas. Right now there is just a simple check to see if the player is in sight and in range, but I would like to add cover seeking when the player does so and add automated cover seeking instead of manually placing the nodes.

9. Sources

• How Flanking systems work? (2019, 15 november). Reddit. Geraadpleegd op 11 april 2022, van https://www.reddit.com/r/gamedev/comments/dwjyab/how_flanking_systems_work/
• S. (2016, 11 november). Flanking and Coordinated Movement – Best Practices? Support Forum. Geraadpleegd op 11 april 2022, van https://forum.arongranberg.com/t/flanking-and-coordinated-movement-best-practices/3273/2
• Flanking maneuver? (z.d.). Unity Forum. Geraadpleegd op 11 april 2022, van https://forum.unity.com/threads/flanking-maneuver.960811/
• AI Flanking. (2016, 27 mei). Reddit. Geraadpleegd op 11 april 2022, van https://www.reddit.com/r/gamedev/comments/4lbx2b/ai_flanking/
• H. (z.d.). GitHub – h8man/NavMeshPlus: Unity NavMesh 2D Pathfinding. GitHub. Geraadpleegd op 11 april 2022, van https://github.com/h8man/NavMeshPlus
• A* Pathfinding Project. (z.d.). Arongranberg. Geraadpleegd op 11 april 2022, van https://arongranberg.com/astar/
• Technologies, U. (z.d.). Unity – Manual: Navigation System in Unity. Unity Documentation. Geraadpleegd op 11 april 2022, van https://docs.unity3d.com/Manual/nav-NavigationSystem.html
• Wikipedia contributors. (2022, 10 maart). Swarming (military). Wikipedia. Geraadpleegd op 11 april 2022, van https://en.wikipedia.org/wiki/Swarming_(military)
• Wikipedia contributors. (2022b, april 1). Flanking maneuver. Wikipedia. Geraadpleegd op 11 april 2022, van https://en.wikipedia.org/wiki/Flanking_maneuver
• The F.E.A.R. AI. (2018, 15 augustus). [Video]. YouTube. https://www.youtube.com/watch?v=KQN3yKYkFmE&t=146s&ab_channel=Totoooo

Foto

• Navmesh in unity. (z.d.). [Foto]. https://docs.unity3d.com/Manual/nav-BuildingNavMesh.html
• navmesh plus. (z.d.). [Foto]. https://github.com/h8man/NavMeshPlus/wiki/HOW-TO
• Example flanking. (z.d.). [Foto]. https://puu.sh/p7eDo/dfb1355633.png
• The Battle of Kirkuk flank. (z.d.). [Foto]. https://en.wikipedia.org/wiki/Flanking_maneuver
• A* pathfinding. (z.d.). [Foto]. https://arongranberg.com/astar/