Tencent Games has unveiled an autonomous first-person shooter (FPS) AI agent architecture that evaluates whether to accept player instructions based on battlefield conditions rather than following them blindly. Jing-Wen Yang, head of the Intelligent NPC Team and Large Language Model Team at Tencent Games' Central Tech AI Engine Division, delivered a presentation on "Building Autonomous AI Agents for FPS Battlefields" at the game developer conference 'gamescom dev' held in Cologne, Germany, on August 24 (local time). The mobile battle royale game Peacekeeper Elite, operated by Tencent in China, was used for the demonstration visuals.

Yang highlighted the core issue: a divide in game AI between "knowing what to do" and "doing it." An FPS agent must handle a real-time changing battlefield, open-ended tasks with no set answers, and free-form player instructions all at once. Rule-based systems offer reliability and predictability, but cannot venture beyond designed scripts. Reinforcement learning (RL) excels at acquiring specific skills, but requires clear tasks and reward functions. Large language models (LLMs) excel at understanding instructions and goal reasoning, but were not built with real-time perception and precise control in mind.

Yang explained that while a language model might determine that it "needs to take the high ground," it does not know how to safely navigate there, when to engage, or how to aim and shoot.

A dual system splitting reasoning and control

The solution chosen by Tencent Games is a dual-layer system splitting the agent into two tiers. A Vision-Language Model (VLM)-based cognitive reasoning system determines "what to do" at a semantic level, while a reinforcement learning-based high-speed execution system handles "how to do it" in real time. When the reasoning system passes down intentions and skill plans, the execution system carries them out, sending feedback and key events back to the reasoning system if circumstances change. The two systems differ in their perceptual information, design architecture, and output format.

Perception in the execution system focuses on geometric information. For vision, a raycast grid combining 7 horizontal and 11 vertical rays—77 in total—scans a field of 120 degrees horizontally and 60 degrees vertically. Six rays behind the character and six in front of its feet handle rear awareness and short-range movement, respectively. Added to this is a multi-layer heightmap dividing surrounding terrain into a grid. The heightmap stores ground elevation, distance to boundaries, and overhead clearance height in separate channels to represent ledges, vertical drops, and open space.

The scope of state information fed into the execution policy was also disclosed. It includes self-information such as health and posture; backpack, equipment, and weapons; door states; throwables; the location and timing of the "safe zone" and "red zone"; sound sources like footsteps and gunshots; whether obstacles can provide cover or be traversed; nearby players and the remaining player count; and network latency.

For the policy network backbone, LSTM was used instead of Transformer. This decision was based on the premise that inference latency dictates performance in real-time FPS control. Spatial and geometric data, such as heightmaps and raycast matrices, are processed by a Convolutional Neural Network (CNN), while numerical states for the agent, teammates, and enemies in line of sight are handled by separate Multi-Layer Perceptrons (MLPs). Since player counts for teammates and enemies vary, a weight-sharing encoder is used. The LSTM, receiving the combined encodings, maintains hidden states to provide short-term memory; relying solely on the current frame causes frequent target switching or unstable reactions when an enemy briefly disappears. A separate attention module is attached for target selection. The final output is divided into continuous values for horizontal/vertical aiming, binary values for shooting and reloading, and multi-discrete values for movement and other actions.

A skill library broken down into combat, driving, and retreat

Training was decomposed into reusable expert skills rather than burdening a single policy with every action. Large-scale RL incurs heavy GPU costs, and requiring one policy to handle all behaviors makes optimization difficult. The training architecture runs multiple simulation environments in parallel while sampling mini-batches from an experience replay buffer.

Details down to reward design were revealed for three skills. The Combat Expert places two agents in arbitrary scenarios, giving +1 point for eliminating the opponent and -0.01 points per step, measuring performance by the time taken to eliminate the target. The Driving Expert has the agent drive a vehicle from a random starting point to a random destination, granting +1 point upon arrival and -0.01 points per step. The Retreat Expert completely disables attacking for the learning agent, permitting only evasion, taking cover, and changing stances, while rewarding +0.01 points per survival step and penalizing -1 point if eliminated. Yang stated that this process relies entirely on simulation-based learning without using human gameplay data.

On the overall architecture diagram, loot, recovery, throwable, and communication experts were also listed as skill library items alongside combat and driving. Specific training designs were presented for the three aforementioned skills.

The reasoning system receives global and local map images along with text-formatted situational reports. The maps mark the safe zone, "blue zone," red zone, vehicle, and supply drop locations, while enemies are categorized into confirmed hostiles and suspected locations.

The situational report includes health, boost, coordinates, altitude, view direction, movement status, and stance as core metrics. Added to these are primary weapon, secondary weapon, and pistol ammo; helmet and vest levels; counts for first aid kits, bandages, painkillers, ammo by caliber, smoke grenades, and hand grenades. It also incorporates rule-calculated game events—such as engagement status, nearby cover availability, whether damage was taken, or gunshots were heard—and current decision states.

Memory is managed across three streams: conversation logs with teammates, a chronological record of decisions, and reflection reviewing past choices. Reflection categorizes well-made decisions, accepted risks, overlooked details, and areas for improvement.

Knowledge is integrated using Retrieval-Augmented Generation (RAG). From a gameplay knowledge base created by game designers, queries based on current gear, player relations, location, and environment undergo vector search and reranking to retrieve only top results—preventing the entire knowledge base from exceeding context limits. Domain knowledge—such as holding a bottleneck to force enemy movement or using smoke grenades to block line of sight—is dynamically selected based on the situation. The system prompt defines safe zone rules, game objectives, agent goals, and behavioral personas (aggressive, cautious, or adaptive).

The model's output is not keyboard or mouse inputs, but "skill chunks"—short sequences linking high-level intentions (like movement, engagement, or searching) with executable skills. The skill planner selects which skill to use while generating parameters such as target, location, duration, and quantity, passing them to the execution system.

Interrupting events, rejected orders

Skill chunks reduce how frequently the reasoning model needs to be called. When a movement decision is made, a sequence—walking to a vehicle and then driving to the destination—is bundled in advance and processed sequentially by the execution system if nothing unusual occurs. Because the VLM does not need to make new decisions every frame, latency demands are significantly alleviated.

What interrupts a plan is an event. If an enemy is spotted during movement, the active skill chunk is canceled, intention shifts to combat, and a new skill chunk chaining shooting and throwables is generated. Tencent Games categorized this as event-driven replanning—a design that preserves efficiency during quiet moments and reacts only when the battlefield shifts.

Even with the same goal, execution paths diverge depending on context. When the final safe zone closed in with uncertain surrounding threats, the agent deployed smoke before entering rather than rushing straight in. When the safe zone was far away and an enemy was encountered en route, the agent abandoned its vehicle acquisition plan, resolved the firefight, and then returned to its original transit plan. When enemies were inside a building with unknown numbers or status, it threw a grenade to apply pressure before entering to clear rooms. When outnumbered, it repositioned under smoke cover, healed, and resumed combat.

Player utterances are handled under the same principle, treated as an input rather than an absolute command. Utterance intent is classified first, filtering out casual chat so it does not affect current intentions or skill chunks, while only tactical utterances—such as requests for help or strategic suggestions—are passed to the model alongside situational reports, gameplay knowledge, and system prompts.

If health is low and nearby buildings are unsafe, a player's call to retreat aligns with tactical reality, prompting the agent to break engagement, find cover, and heal. Conversely, if ordered to find a vehicle while actively engaging an exposed enemy at close range, the agent refuses the order and continues combat. Yang stated that the agent listens to players, but does not follow them blindly.

Yang cited three remaining challenges: behavioral consistency, ensuring language, intent, and action align so players can read the agent's intent and reasoning; adaptive planning, updating only relevant portions of a plan when battlefield conditions shift rather than replanning from scratch; and lifelong learning, expanding the skill library when new systems or maps are added without retraining the entire model.

The research was conducted by Tencent Games' GIGA (General Instructable Game Agents) team, which aims to develop agents capable of perceiving virtual worlds and following open-ended instructions.