I spent some time thinking about it, but I'm aware of only two main meanings. There might be more that aren't coming to me right now though...
In local search problems or sometimes in optimization for machine learning, the "topology" of a problem corresponds to the change in the function you're optimizing as you move between adjacent states. If the change is sharp, you have a "rugged topology". If it's gentle and continuous, you have a "smooth topology". See on page 2 of An Introduction to Fitness Landscape Analysis and Cost Models for Local Search for example.
The other major meaning is with reference to the structure (topology) of a combinatorial graph. Many modern machine learning algorithms are based in the idea of combinatorial graphs, including Bayesian Networks, Sum/Product Networks, and Deep Neural Networks. Here, topology refers to the topological ordering of a directed graph, or more informally, to "how the graph is structured". For example, in a neural network, the depth and width of the network's layers, and the nature of the connections between layers, define the topology of the network.
Additionally, it gets used a lot in the sense of the second meaning in other areas of AI, just because those areas also use graphs. For example, in automated planning, or in probabilistic reasoning, it is also common to represent your problem as a combinatorial graph. You could then talk about the "topology" of the problem.
