2
$\begingroup$

I am looking for the standard notation to define element-wise / Hadamard-style functions, if there is one.

That is to say, if the operator I am looking for were represented by a hexagon ⬡, I could use it as such:

$$A(x) = \underset{i}{\Large{⬡} } f(x_i)$$

$$A : \mathbb{R}^n \rightarrow \mathbb{R}^n$$ $$f : \mathbb{R} \rightarrow \mathbb{R}$$

It is very convenient to define such functions explicitly because I want to manipulate them: $B \circ A$ . It seems to me that the following notation is correct: $A_i(x) = f(x_i)$ but I worry it is nonstandard and confusing.

My functions are non-linear so I cannot simply apply them directly to the array as a vector.

As stated in an answer, this is unnecessary when a function is strictly scalar because it is implied to apply element-wise. There are still some situations I would hope to have it:

$$\underset{ij}{\Large{⬡} } e^{M_{ij}} \ne e^M$$

The answers suggest to me the best option would be something like these:

$$A(M) = \text{for each } i,j: e^{M_{ij}}$$ $$A(M) = \text{element-wise}: e^{M_{ij}}$$

The question is now closed in the negative, but I would welcome a new answer. Would be nice to find something like $\forall$.

Related:

$\endgroup$
0

2 Answers 2

1
$\begingroup$

Based on my experience, I would say that the standard notation is just to have a regular function, and specify that it applies element wise. For example, a common notation for activation functions is $\sigma$, so e.g. you could represent the activations of a regular dense layer as $\sigma(W x + b)$ where $x, b$ are vectors and $W$ is a matrix. I've never seen a special notation for specifying that the function $\sigma$ is applied element wise.

As you suggest in the question, if the function to be applied element wise is linear, then you can use either hadamard product, e.g. $a \circ x$ or the diag function $\text{diag}(a)x$.

$\endgroup$
2
  • $\begingroup$ Thank you, great answer, and this does match what I have seen in the literature: a scalar function applied to a vector is implicitly element-wise. Thinking about it, I can now see why there isn't an operation like the one I asked for. There are still some cases where it would be useful (updated question), so I'll leave the question open on the slim chance there has been a notation for it somewhere. $\endgroup$
    – user7834
    Oct 23, 2021 at 9:09
  • $\begingroup$ Thinking more sensibly, I think the "operator" is the just words "element wise". $\endgroup$
    – user7834
    Oct 23, 2021 at 9:31
1
$\begingroup$

The mathematical notation for complex tensorial expressions always tries to balance complexity and precision. More precise notation - the one that explicitly spells all the indices - becomes extremely convoluted very quickly. My favorite example illustrating it is from physics -- the Standard Model Lagrangian is written shortly on T-shirts and coffee mugs as:

$$ \mathcal{L} = - \frac{1}{4} F_{\mu \nu} F^{\mu \nu} + (i \bar{\psi} \hat{D} \psi + \bar{\psi}_i y_{ij} \psi_j \phi + h.c.) + |D_\mu \phi|^2 - V(\phi) $$

But if you try to expand all the indices in all the objects above - then it barely fits on a page.

On the other hand, more succinct notation always leads to ambiguities in interpretations. Your example $f(x_i)$ can be read as: $$f(x_0, x_1, \dots, x_N)\quad \text{or as}\quad f(x_0),f(x_1), \dots, f(x_N)$$
One way to implicitly resolve this ambiguity is to show that the index $i$ "escapes" the argument brackets:

$$a_i = f(x_i)\quad \text{or e.g.} \quad \sum_if(x_i)$$

This can only be interpreted as $f(x_i)$ being element-wise. Also, at least in my opinion, using $x_i$ with index and $x$ without index in the same expression is extremely confusing.

And, of course, the best way to resolve these ambiguities is to state them explicitly. For example, I've seen authors using square brackets $f[x_i]$ or capital letters $F(x_i)$ the vector-argument functions.

$\endgroup$
2
  • $\begingroup$ Hi Kostya thank you for your answer. Sorry, I think my question may not be clear, f(x_i) is not a vector function, it is a simple scalar function. f: R->R ; a: R^n->R^n . Unless I miss something, I have to have a variable index and unindexed in the same expression, because I am relating behaviour an the variable to its components? If I just write a_i = f(x_i) it looks like x is a free variable. $\endgroup$
    – user7834
    May 19, 2021 at 19:25
  • $\begingroup$ I have edited my question to follow your advice regarding capital letters. $\endgroup$
    – user7834
    May 19, 2021 at 19:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .