In what ways is the term “topology” applied to Artificial Intelligence?

Question

I have a only a general understanding of General Topology, and want to understand the scope of the term "topology" in relation to the field of Artificial Intelligence.

• In what ways are topological structure and analysis applied in Artificial Intelligence?

Answer 1

I spent some time thinking about it, but I'm aware of only two main meanings. There might be more that aren't coming to me right now though...

In local search problems or sometimes in optimization for machine learning, the word "topology" of a problem is used to refer to the change in the function you're optimizing as you move between adjacent states. If the change is sharp, you have a "rugged topology". If it's gentle and continuous, you have a "smooth topology". See on page 2 of An Introduction to Fitness Landscape Analysis and Cost Models for Local Search for example.

(Note that texture, curvature, and other morphological features lacking a discontinuity are not topology in the canonical mathematical sense. The geological term "topography" may have been a more appropriate choice.)

The other major meaning is with reference to the structure (topology) of a combinatorial graph. Many modern machine learning algorithms are based in the idea of combinatorial graphs, including Bayesian Networks, Sum/Product Networks, and Deep Neural Networks. Here, topology refers to the topological ordering of a directed graph, or more informally, to "how the graph is structured". For example, in a neural network, the depth and width of the network's layers, and the nature of the connections between layers, define the topology of the network.

Additionally, it gets used a lot in the sense of the second meaning in other areas of AI, just because those areas also use graphs. For example, in automated planning, or in probabilistic reasoning, it is also common to represent your problem as a combinatorial graph. You could then talk about the "topology" of the problem.

Answer 2

The General Meaning

The idea of topology in mathematics has been extended to networks by many, both in academia and in large web service provider business. Whether in its original geometric context or in its more recent use in network architecture (in the widest sense of the word network), it delineates two classes of system characteristics.

• NOT a topological alteration — That which can be altered by stretching, bending, twisting, and otherwise morphing the features without altering the higher level characteristics of their interrelationship
• Topological alteration — That which goes beyond modifying magnitude or curvature to alter the nature of how the lines and surfaces join and interrelate.

Introduction to Children Using 3D Geometry

Many younger students learn of topology through the introduction to a Möbius strip. The strip can be constructed with paper, scissors, and tape in math class. The teacher will point out that, unlike a wrist band made with the same strip of paper, the Möbius strip is one-sided because of the 180 degree twist before applying the tape. Without the twist, it is two-sided.

Twisting before joining creates a topological change. Changing the length or width of the strip does not.

General Meaning in Network Connectivity

With network topology, in the context of web service provision, the system involved is discrete. The items in a network diagram can be represented as squares or elongated rectangles. Changing the height-to-width ratio of a rectangle representing a load balancer (for example) means nothing in terms of functionality. In that architectural context, adding a 12th web server to a bank of 11 web servers is not a topological change. It is varying the size of an array, which, in discrete systems is the only possible meaning of stretching.

Add a switch to a bank of switches and you've morphed the architecture but not changed the way the banks of devices interconnect and perform their functions relative to one another. All of the same capacity planning formulae still apply.

Conversely, of one adds a bank of hardware firewalls between the web servers and the switches, when there were no firewalls there before, the topology has changed. All the capacity planning formulae must be reviewed and adjusted. Analysis cannot be re-figured by simply changing a numeric value in an existing equation.

It is the same with an electrical circuit. Although component values are on a continuous real number scale, the interconnections between components are discrete. Add a fixed resistor to a series of resisters in a circuit, and the total resistance is simply the sum. The network behavior is not changed other than to change the number representing the total resistance. Less current passes for a given voltage drop. The circuit topology has remained constant.

Conversely, if one or more capacitors are added between signal and signal return (ground), the nature of the circuit changes. The result is a low pass filter, since the high frequencies are met with no resistance in the capacitive shunt to ground. The equations modelling the circuit are entirely different. The topology of the circuit has changed.

Machine Learning Context

In the context of machine learning, we have discrete systems again. The system is not just a discrete (digital) system in terms of the numerical data passing through the system as signals, but also in the continuity of the activations. In other words, there is an array of activations represented as a layer, not a continuous and monolithic block activation. The algorithm to perform the activation is reused and each element in the array of activation outputs is independent. The layer in every important way is discrete.

There is no functional meaning to stretching, bending, or twisting any of the features of an artificial network. The activation functions, the attenuation matrices that are multiplied with the input vector to produce the weighted values input to the activation functions, and the lines used to represent those connections can be stretched or bent in the images used to graphically represent the network, but it has no meaning in terms of the mathematics of the network or the algorithm to realize it.

The only possible characteristics in the network architecture that would NOT be topological are those characteristics that can be, in discrete terms, stretched or contracted. These would be the number of elements in the parameter matrix or convolution kernel, the number of activations in an array representing a layer, or some other array size that determines the signal dimensions.

• Number of activations in the layer
• Horizontal, vertical, pixel depth of image throughput
• Capacity of frames considered in temporal learning
• Size of the integer, fixed, or floating point numbers used
• Number of like-sized middle layers somewhere in the network

These are things that change the behavior of the network but in ways that do not change the character of governing equations. They are NOT topological changes. These are topological changes.

• Changing an activation function type, not just in terms of a hyper-parameter but in the nature of the function
• Adding an activation layer unlike the one before it or after it
• Connecting two separately functioning networks in a collaborative way to accomplish a goal, even if the networks create a balance through adversarial interaction, as in the case of GAN design
• Adding or removing a rotational layer or conventional Cartisian convolution layer in series with activation layers

In General Across AI

One of the most challenging and interdisciplinary applications of artificial intelligence is in control systems for devices with physical functionality and response. This may be in any number of application categories.

• Factory robotics, where adaptation to machine tool wear is unpredictable and AI is used to maximize the quality of manufacturing output
• Automated vehicles, whether aeronautic or road vehicles
• Robots used to demonstrate where AI is in terms of simulating humanity — and may someday soon reach a near Asimovian I-Robot level of simulation
• Robotic devices to clean house or farm, with AI components to recognize changes in the topology of the task and adapt — That's another use of the term topology where the physical interrelationships of trees, stones, ditches, tables, couches, and chairs may have changed.

Such systems require an interconnection of overall system components that form a topology.

System Architecture and Topology

Now is a good time to distinguish two things.

• ARCHITECTURE
• TOPOLOGY

Although the argument for considering topology a subset of architecture is strong, declaring them equivalent is quite incorrect. There are dozens if not hundreds of architectural concerns that have little or nothing to do with topology. These are just a few.

• Capacity planning when scaling a cluster
• Determination of network throughput and latency minima
• Specifying memory and disk space
• Determining throttling
• Cost containment
• Selection of third party products based on ROI
• Development of deployment policy
• Full life cycle configuration management
• Evaluating development tool options
• Determining inches of rack space
• Selection of CAT-7 vs CAT-6

One could also consider items in topology that have nothing to do with architecture. Functional minutia, such as how to normalize input signals or otherwise clean inputs of noise and outliers is topological but not architectural. Whether data hygiene is needed is architectural not topological. There is some overlap, but not much.

Back to Robot Design

In the topological (as opposed to architectural concerns) of robot experimental and theoretical research and marketable product development, the topology is the first thing to consider after defining what the robot is expected to do.

In this context, the topology is the control topology, with absolutely necessary complexity in interconnection of banks of like items. How many items are in the bank is an architectural decision, but at this topological segment of R&D, it is assumed that the banks can have N elements of a particular type. Here are a few elements that are likely to be required.

• Data acquisition components
• Filters
• Adaptive components requiring artificial intelligence capabilities to modify rule sets, alter parameters of fuzzy logic, train networks, or apply Bayesian methods to stored observations
• Parallel systems to preemptively detect safety issues and override
• Closed loop feedback — key in many systems to ensure accuracy of position, thermodynamic conditions, or other things that must remain in stasis for proper continued operation
• Servo or stepper motor drivers
• Linear or radial encoders
• Heaters, coolers, valves, or fluid conditioners

There is rarely a series connectivity that could be represented in a sequence, which is perhaps the most trivial high level topology. There are instead ...

• Closed loops (circles or bands of multiple channels)
• Signaling in the form of vectors, matrices, cubes, and hyper-cubes (this last in the case of computer vision requiring action detection)
• Multiple upstream components leading to an aggregated function
• Distribution of the signals of a single source to a wide variety of heterogeneous components
• Banks or grids of components

The topological design of a nuclear submarine or power plant is like this. The topology must be determined after the system requirements are known but before any other high level design can be reliably determined.

In Summary

Topology is a representation of structure absent of size or curvature considerations, and this is true for all uses in math, science, and technology.

Distortions in the meaning of the term to accommodate jargon generated by misuse sometimes found in machine learning or AI literature should be avoided. Topology is too important a concept in the research and development of complex systems to obfuscate.