Topic: Automated Testing for Complex Games: Developing a Custom Auto-Play Test Tool for 'The First Descendant'

Speaker: Kang Jang-hoon - Engineer, Nexon Games Magnum Studio

Field: Programming

Recommended Audience: Programmers, Tech Leads, and TA/QA Engineers interested in QA automation

Tags: #Programming #Automated #QA Testing

[🚨 Lecture Topic] This session covers the development of 'M Auto-Play Tester,' a proprietary automated performance testing tool built directly into the game package to overcome console platform security restrictions. It is tailored to the complex terrain and diverse combat environments of 'The First Descendant.' The talk shares key pipeline processes for unmanned automated testing, including a 3-stage fallback pathfinding system and timeout guards to prevent bots from getting stuck, smart combat caching to prevent frame drops, and Levenshtein distance matching to handle typos flexibly. It also covers know-how on visualizing performance data based on spatial coordinates.

The Motivation Behind Building an Auto-Play Test Tool for 'The First Descendant'

While automated systems are often colloquially referred to as a simple 'click,' building them requires significant cost. Kang Jang-hoon, an engineer at Nexon Games who worked on 'The First Descendant,' stated, "The core of automation is not perfect reproduction, but creating a process that never stops no matter what happens." He shared the story of how the team built their auto-play test tool.

Engineer Kang explained that the decision to develop an automated play-testing tool for The First Descendant was driven by the sheer number of platforms and the resulting combinations of options. He noted that it is physically impossible for humans to manually test every scenario in such an environment. He also pointed out that security restrictions on console platforms prevent the use of external play-testing tools. Consequently, the team behind The First Descendant chose to develop an automated play tool that could function from within the game package itself.

From Fallbacks to Architecture and Data Visualization

Engineer Kang highlighted four key elements of the auto-play tool: fallbacks, architecture, data visualization, and more.

Discussing pathfinding with a fallback structure, he noted that the first hurdle in automated testing is reaching a destination within a complex map, as bots often get stuck in terrain, halting the test. To prevent this, the development team built a 3-stage pathfinding system.

In the first stage, 'Standard Navigation,' the bot attempts to use basic mesh navigation. In the second stage, 'Detour Navigation,' the bot avoids areas where it frequently gets stuck. Finally, in the third stage, 'Flat-Ground Navigation,' the bot ignores Z-axis changes to find the safest flat path, automating the movement of the test bot.

He added that if the 3-stage process fails, a 'timeout guard' triggers a teleport function to rescue the bot from being stuck.

Engineer Kang then mentioned 'smart combat caching' as a key technology to prevent frame drops. Given that 'The First Descendant' features many monsters, he explained that if a bot scans all monsters every tick, it causes severe GPU overhead and frame drops.

Based on this, the team leveraged the fact that monsters are drawn to the player via the aggro system. They only perform a full scan when the cache is empty and cache only the nearest few monsters in an array. The key was eliminating unnecessary calculations by updating distances and sorting priorities only for the cached targets.

Third, he emphasized that they built the system to continue automation despite typos using Levenshtein distance. For a test bot to perform a mission, it must accurately identify the assigned task; if a developer makes a typo, the bot might stop. This was prevented in advance.

Engineer Kang mentioned that by introducing the 'Levenshtein distance' algorithm, they added flexibility, allowing the system to recognize a mission as identical if the similarity between the game data and the assigned mission is 75% or higher. This allows the bot to perform its duties even if there are minor errors in the data.

Finally, the development team adopted a modular structure to ensure that all logic, from pathfinding to performance data collection, remained decoupled. They increased reliability by resetting data every test cycle and, through internal overhead measurements, kept test interference under 1.1ms.

Through this system, they were able to pinpoint exactly which platform, setting, and level segment caused bottlenecks, going beyond simple frame drops. By optimizing based on this data, they achieved a significant reduction in frame time for the worst 5% of spike segments, from 20ms to 11ms.

The human role is also important; try taking on automation

In closing, Engineer Kang stated that automation tools cannot completely replace humans. While bots are diligent at collecting data, they cannot catch 'visual errors' like lighting or texture streaming delays. He emphasized that the answer to automation is not replacement, but supplementation—while machines handle repetitive tasks, humans should focus on higher-level judgment.

He also expressed interest in introducing automatic screenshot comparison, automatic test video recording, and mission strategy modularization in the future. He concluded by encouraging developers to try building their own tools with a 'let's just do it' mindset rather than waiting for a perfect design.