In the literature both concepts (NPC and AI) are introduced as AI systems. The idea is that, the computer will act autonomously. But it's naive to guess that creating a non player character is equal to the task of a creating a human-replacing AI. The difference isn't located in the technology which is working in both cases with Finite state machines, but it has to do with the difference between an environment and an agent. To make it short: it's an easy task to create a NPC, but it's hard or even impossible to create an AI which is playing alone.
From the history of computer games the first known example for a NPC was given in the pacman game. The NPC was equal to the ghost control system which is attacking the player. On the first look the game developers have made a good job, the ghost NPC was able to defend the maze and made the play for the human player more demanding. But can we take the same technology for simulating also the player? No it's not possible. The pacman game contains of two different individuals. A player who has to win the game, and a maze which is trying to defend itself.
Let us go a step backward and assume that a normal game is played by a human. He has the control of a figure in the game and takes the decision. What is possible from a technical point of view is to make the game more interesting and invent some NPC characters. The task of these agents is to make the game more difficult. They are not acting as players in the game, but they are equal to the game. Creating such NPC characters isn't very demanding. In the easiest case a Finite state machine is well working. Such a engine works with if-then-condition, e.g. “if player in reach then attack”.
The surprising aspect is, that the same technology isn't able to emulate the human player. The game will need always a human in the loop. This is important to know because in classical engineering the goal is quite opposite. Engineers are trying to reduce the workload for humans. A washing machine was invented to make life more easier but not more complicated. The problem is, that a NPC is doing the opposite. He won't improve the situation but make it worse. The NPC is by definition an entity who is working against the human. He makes the level more challenging, he lowers the productivity of the human. The interesting point is, that this behavior is the only one which is available. If an AI is equal to a NPC, this AI will work against the human player.
Let us analyze the situation from an engineering perspective. Engineers are trying to build system which have a better productivity. The idea is to make the life easier for the human. This can be done not by introducing a NPC into the game, but by removing it. If we are switch off the NPC ghosts in pacman the human will find the points more easier. This sounds a bit paradox, because the common assumption is that an AI is equal to something who helps the human.
From the perspective of sociology the phenomena is called productivity paradox. It means, that after introducing robots on the workplace, the human workload is higher than before. Does this make sense? Yes, if the assumption is, that the robot is always a NPC which is working against the human. This will direct us to an important question: Is it possible to create an AI who isn't working as a NPC but on the same side like the human?