Skip to content
Home » Blog » How to Use Queue Based State Machine in Unreal Engine (C++)

How to Use Queue Based State Machine in Unreal Engine (C++)

Hello my friends. Here you go again. Today I will talk about Queue based State Machine which is a decision making algorithm. I have mentioned Behavior Tree before. You can go to that article by clicking here.
A state machine is an algorithmic model used to manage decision-making processes by transitioning between a fixed set of defined states. Moreover, Each state encapsulates specific behavior, and transitions between these states occur based on events or conditions. For example, in a game, a character might have states like “idle,” “running,” and “jumping.” Depending on player input or other in-game events, the character will transition from one state to another. State Machines are widely used not only in game AI but also in animation systems, robotics, and many other fields where clearly defined behavior is crucial.

Why Use a State Machine?

  • Modularity and Clarity:
    By breaking down a system’s behavior into separate, manageable states, you create modular code that is easier to understand and maintain. Each state is responsible for its own logic, and transitions are clearly defined.
  • Error Prevention:
    Since the system can only exist in one state at a time, there’s no risk of conflicting behaviors (for example, a game character trying to run and jump simultaneously). This containment helps reduce bugs and simplifies debugging.
  • Flexibility and Scalability:
    A well-designed state machine can be easily extended. If you need to add new behaviors, you simply define additional states and specify the transitions—without having to overhaul the entire system.
  • Performance:
    When implemented efficiently (for example, using a simple switch-case structure), FSMs offer low-overhead solutions that work well in real-time applications, such as games and interactive animations.

Implementing State Machine

This class is the foundation of all our AI states. Each state has three main methods:

  • StartState: Runs when the state starts. (Initial operations, setting necessary variables, etc.)
  • TickState: Runs every frame. (Continuing operations, motion, animation updates, etc.)
  • EndState: Runs just before the state ends. (Cleanup or exit operations.)

and bIsDoneState : whether the current state is over or not.

UCLASS(Abstract, Blueprintable)
class YOURPROJECT_API UAIState : public UObject
{
    GENERATED_BODY()

public:
    // whether the current state is over or not
    bool bIsDoneState;
    // Called when the state is started. Owner can indicate the controls that the state controls.
    virtual void StartState(AAIController* Owner) { }

    // Called on every tick.
    virtual void TickState(float DeltaTime) { }

    // Called just before the state is terminated.
    virtual void EndState() { }
};

This class holds the current (active) state of the AI ​​and manages state transitions. At each tick, the TickState function of the active state is called. Also, when a transition to a new state is desired, the EndState method of the current state is called. Then the StartState method of the new state is executed.

  • InitializeStateMachine() -> initializes the owner controls.
  • ChangeState() -> Used to switch to a new state. If the current state exists, it runs its EndState function and then runs the start function of the new state.
  • Tick() -> In the Tick function, it runs the Tick function of the currentState.
UCLASS(Blueprintable)
class YOURPROJECT_API UAIStateMachine : public UObject
{
    GENERATED_BODY()

public:
    // Initializes the State Machine with a controller.
    void InitializeStateMachine(AAIController* Owner)
    {
        Controller = Owner;
    }

    // Transitions to a new state.
    void ChangeState(UAIState* NewState)
    {
        if (CurrentState)
        {
            CurrentState->EndState();
        }
        CurrentState = NewState;
        if (CurrentState)
        {
            CurrentState->StartState(Controller);
        }
    }

    // The tick function is called in each frame.
    void Tick(float DeltaTime)
    {
        if (CurrentState)
        {
            CurrentState->TickState(DeltaTime);
        }
    }

private:
    // The current active state.
    UPROPERTY()
    UAIState* CurrentState = nullptr;

    // The controller that the State Machine controls.
    UPROPERTY()
    AAIController* Controller = nullptr;
};

Now let’s create a PatrolState by extending the AIState class we created. This state will allow our AI to travel between given points. We override the StartState, TickState and EndState functions from the AIState class. We also define the variables we will use.

  • PatrolPoints -> Array to store patrol waypoints.
  • CurrentPointIndex -> Index of the current patrol waypoint.
  • AcceptanceRadius -> The acceptable distance (in cm) to consider a waypoint as “reached”.
UCLASS()
class YOURPROJECT_API UPatrolState : public UAIState
{
    GENERATED_BODY()

public:
    // Called when the state starts.
    virtual void StartState(AAIController* Owner) override;

    // Called every tick while the state is active.
    virtual void TickState(float DeltaTime) override;

    // Called before the state ends.
    virtual void EndState() override;

protected:
    // Array to store patrol waypoints.
    TArray<FVector> PatrolPoints;

    // Index of the current patrol waypoint.
    int32 CurrentPointIndex = 0;

    // The acceptable distance (in cm) to consider a waypoint as "reached".
    float AcceptanceRadius = 50.0f;

    // The actor controlled by this state.
    UPROPERTY()
    AAIController* OwnerController = nullptr;
};

Now let’s code the functionality of our PatrolState class. We initialize the necessary variables in StartState and set the first patrol point to where our AI will go.

In TickState, we check whether we have reached the patrol point to be reached. If we are as close as “AcceptanceRadius”, we move on to the next point and assign the new patrol point to our AI as the target point.

Finally, in EndState, we print a log. In this way, we understand whether the State has ended or not. And set finish state bool.

void UPatrolState::StartState(AAIController* Owner)
{
    UE_LOG(LogTemp, Warning, TEXT("Patrol State started."));
    
    bIsDoneState = false;
    OwnerController = Owner;
    CurrentPointIndex = 0;
    
    if (!OwnerActor || PatrolPoints.Num() == 0)
    {
        return;
    }
    // Update blackboard MoveLocation key with first point
    OwnerController.GetBlackboardComponent()->SetValueAsVector("MoveLocation", PatrolPoints[CurrentPointIndex]);
}

void UPatrolState::TickState(float DeltaTime)
{
    if (!OwnerActor || PatrolPoints.Num() == 0)
    {
        return;
    }

    FVector CurrentLocation = OwnerActor->GetActorLocation();
    FVector TargetLocation = PatrolPoints[CurrentPointIndex];

    // Check if the actor has reached the target waypoint.
    float Distance = FVector::Dist(CurrentLocation, TargetLocation);
    if (Distance < AcceptanceRadius)
    {
        // Move to the next waypoint, looping back to the first one if at the last waypoint.
        CurrentPointIndex = (CurrentPointIndex + 1) % PatrolPoints.Num();
        // Update blackboard MoveLocation key with next point index
        OwnerController.GetBlackboardComponent()->SetValueAsVector("MoveLocation", PatrolPoints[CurrentPointIndex]);
    }
}

void UPatrolState::EndState()
{
    UE_LOG(LogTemp, Warning, TEXT("Patrol State ended."));
    bIsDoneState= true;
    // Cleanup operations can be performed here before transitioning out of the state.
}

This code snippet shows how the AI ​​Controller will control AI behaviors using state machines throughout its lifecycle. Initially, the state machine is created and initialized, then the AI’s state (i.e., Patrol) is set. In each frame, the Tick function of the state machine is called to execute the current behaviors of the AI ​​based on the current state.

With this structure, you can manage AI behaviors in a modular manner, add new states, and provide dynamic transitions between existing states.

AMyAIController::AMyAIController()
{
    PrimaryActorTick.bCanEverTick = true;

    // Create the State Machine object.
    AIStateMachine = NewObject<UAIStateMachine>(this);
}

void AMyAIController::BeginPlay()
{
    Super::BeginPlay();

    // Initialize State Machine.
    if (AIStateMachine)
    {
        AIStateMachine->InitializeStateMachine(this);

        // For example, switch to Patrol state
        UPatrolState* PatrolState = NewObject<UPatrolState>(this);
        AIStateMachine->ChangeState(PatrolState );
    }
}

void AMyAIController::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);

    // Call the Tick function of the State Machine
    if (AIStateMachine)
    {
        AIStateMachine->Tick(DeltaTime);
    }
}

Adding Queue Job System

In this paragraph, It lets you schedule multiple tasks (or states) so that they execute one after another. Instead of having one state interrupt another or losing tasks if multiple commands are issued at once, the queue holds each state in order and processes them sequentially. This is especially useful when an AI needs to perform a series of actions—like patrolling, attacking, and then retreating—without any steps being skipped.

By separating the moment an event is triggered from when the corresponding state is actually processed, a queue system decouples event generation from execution. This means the AI can receive and store several commands while it’s busy with the current task, and then process them one at a time, ensuring smoother transitions and more predictable behavior.

With a queue in place, you can easily add, remove, or reorder tasks. This makes your AI system more adaptable to complex scenarios where priorities may change dynamically. For instance, if an emergency task arises while the AI is following a routine path, the queue can adjust to handle this new, higher-priority task without discarding the current sequence.

  • InitializeStateMachine: Sets up the AI state machine with the actor it controls.
  • EnqueueState: Queues up a state for later execution, ensuring multiple tasks can be lined up.
  • ChangeState: Immediately transitions to a new state, ending the current state and starting the new one.
  • ProcessQueue: Ends the current state and starts the next state from the queue, allowing for sequential processing.
  • Tick: Continuously updates the active state every frame so that state-specific logic is executed consistently. And most importantly section is “If the current state is over, we move on to the next state.“

In other words, This queue-enabled design makes your AI more flexible and robust, as it can handle multiple tasks in an organized sequence while still allowing for immediate transitions when necessary.

UCLASS(Blueprintable)
class YOURPROJECT_API UAIStateMachine : public UObject
{
    GENERATED_BODY()

public:
    // Initializes the State Machine with an actor.
    void InitializeStateMachine(AActor* Owner)
    {
        ControlledActor = Owner;
    }

    // eni Adds the state to the queue.
    void EnqueueState(UAIState* NewState)
    {
        if (NewState)
        {
            AIStateQueue.Enqueue(NewState);
        }
    }

    // To change the state immediately (optional use).
    void ChangeState(UAIState* NewState)
    {
        if (CurrentState)
        {
            CurrentState->EndState();
        }
        CurrentState = NewState;
        if (CurrentState)
        {
            CurrentState->StartState(ControlledActor);
        }
    }

    // Checks the queue; if the current state is complete (or manually terminated),
    // starts the next state of the queue.
    void ProcessQueue()
    {
        // If there is an existing state, we finish it.
        if (CurrentState)
        {
            CurrentState->EndState();
            CurrentState = nullptr;
        }

        // Get the next state from the queue.
        UAIState* NextState = nullptr;
        if (AIStateQueue.Dequeue(NextState) && NextState)
        {
            CurrentState = NextState;
            CurrentState->StartState(ControlledActor);
        }
    }

    // The tick function is called in each frame.
    void Tick(float DeltaTime)
    {
        if (CurrentState)
        {
            CurrentState->TickState(DeltaTime);
            
            // If the current state is over, we move on to the next state.
            if (CurrentState->bIsDoneState) { ProcessQueue(); }
        }
    }

private:
    // The current active state.
    UPROPERTY()
    UAIState* CurrentState = nullptr;

    // The actor controlled by the State Machine.
    UPROPERTY()
    AActor* ControlledActor = nullptr;

    // Queue: queued states.
    TQueue<UAIState*> AIStateQueue;
};

Finally, come to try state machine with queue job system. As we did before, we use our AIController class to control our state machine. Firstly, We create a stateMachine object in our constructor function. Secondly, use the functionality of our state machine using this object and initialize our state machine in the BeginPlay function and create a few states and add them to the queue with the “EnqueueState” function. Thirdly, we call the “ProcessQueue” function to run the first state and the first state added to the queue starts running. Finally, we activate the tick feature of the states by running the Tick function of the State machine in the Tick function.

AMyAIController::AMyAIController()
{
    PrimaryActorTick.bCanEverTick = true;

    // Create the State Machine object.
    AIStateMachine = NewObject<UAIStateMachine>(this);
}

void AMyAIController::BeginPlay()
{
    Super::BeginPlay();

    if (AIStateMachine)
    {
        AIStateMachine->InitializeStateMachine(this);

        // Add some states to the queue.
        UPatrolState* PatrolState= NewObject<UPatrolState>(this);
        AIStateMachine->EnqueueState(PatrolState);

        UAnotherState* AnotherState = NewObject<UAnotherState>(this);
        AIStateMachine->EnqueueState(AnotherState);

        // Process the queue to initialize the first state.
        AIStateMachine->ProcessQueue();
    }
}

void AMyAIController::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);

    if (AIStateMachine)
    {
        // Call the tick of the active state.
        AIStateMachine->Tick(DeltaTime);
    }
}

In conclusion, I covered how to create an artificial intelligence (AI) state machine using Unreal Engine and C++ and how to provide multiple state management by adding a queue feature to the existing structure. Based on the UAIState, UAIStateMachine and UPatrolState classes, I explained the integration of the state machine via the AI ​​Controller step by step. In the second part of the article, I detailed how different behaviors can be queued and dynamic transitions can be performed thanks to the queue mechanism added to the state machine structure. Above all, These improvements make in-game AI behaviors more flexible, modular and manageable, allowing you to provide advanced AI control in your projects. I hope you enjoyed reading my blog article and learned something.

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *