I decided to practice applying object recognition with TensorFlow for an interesting application this weekend. The application I chose was to recognize enemies in a game world, and as more of a challenge, recognize terrain features and use a pathfind algorithm to navigate past obstacles without running into walls. I'm interested in recognizing the types of enemies (easy to do) with live object recognition, but that terrain recognition concept seems harder.
So I want to ask about the viability of using a supervised learning object recognition model for this or if there's a different kind of AI to apply toward this challenge. Consider this game world frame for Path of Exile:
The lighting is diverse and the world is obviously very complex. So I'm considering creating a specific model for the target area to navigate and either:
Train the model to recognize what traversable floor looks like (in which case I also have to train it to recognize enemy corpses so I can walk over them).
Train it to recognize what the boundary walls of the world look like (but I'm skeptical about this because they're very diverse and I don't think traditional object recognition models will work for this).
My experience so far is improving pre-trained models from the TensorFlow detection zoo repo: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md
What problems and solutions should I consider when approaching this problem from an AI perspective?