Are the domains of objective functions in AI always equals to $\mathbb{R}^D$ or subset of it?

Question

Consider the following paragraph from the chapter named Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.

Central to this chapter is the concept of a function. A function $f$ is a quantity that relates two quantities to each other. In this book, these quantities are typically inputs $x \in \mathbb{R}^D$ and targets (function values) $f(x)$, which we assume are real-valued if not stated otherwise. Here $\mathbb{R}^D$ is the domain of $f$, and the function values $f(x)$ are the image/codomain of $f$.

we can notice that the textbook is taking $\mathbb{R}^D$ as the domain for objective functions. I want to know whether it is valid in general cases.

Do the objective functions that we generally use in artificial intelligence have $\mathbb{R}^D$ as the domain?

I am guessing it would not be since the loss functions are generally defined on the datasets which can also have discrete attributes and hence the objective function cannot be defined on every point in $\mathbb{R}^D$. So, I am guessing that the correct form of the bold statement from the quoted paragraph is "Here the domain of $f$ should be a subset of $\mathbb{R}^D$" if we are intended to deal with a general case. Am I correct or is there any arrangement such as defining $f$ as zero where the function is not defined?

Answer 1

I think, that $\mathbb{R}^{D}$ is most natural choice in practical situations since many kinds of data can be described in a this way:

• Image is 2d array $H \times W$ with each pixel taking value in $\mathbb{R}^{c}$ (say, $c=3$) or a continuous subset of $\mathbb{R}^{c} = [-1, 1]$
• In the sequence modeling problems one gives an embedding vector to each token in some $\mathbb{R}^{k}$. There is no reason priori to put constraints on the embedding vector (probably one may want to clip norm of these vectors)

However, for the applications in physics, or any other fields, by the virtue of the task arbitrary input in $\mathbb{R}^{D}$ may not make sense, and one can be restricted to consider functions only on a certain manifold $M$ (sphere $S^d$, hyperbolic spaces)

In applications of Quantum Machine Learning input is a vector in a Hilbert space - system of quits, whatever. You may be interested to consult this review.