5
$\begingroup$

I am reading the paper Hierarchical Attention-Based Recurrent Highway Networks for Time Series Prediction (2018) by Yunzhe Tao et al.

In this paper, they use several times the expression "semantic levels". Some examples:

  • HRHN can adaptively select the relevant exogenous features in different semantic levels
  • the temporal information is usually complicated and may occur at different semantic levels
  • The encoder RHN reads the convolved features $(w_1,w_2,···,w_{T−1})$ and models their temporal dependencies at different semantic levels
  • Then an RHN is used to model the temporal dependencies among convolved input features at different semantic levels

What is the semantic level?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to convey the message. Here is an interesting picture I found on the internet to illustrate my point. enter image description here

EDIT: I took some time reading the paper. I think "semantic levels" refers here to the different neural networks layers used for the exogenous features.

Here is a modified version of their figure I've drawned to make it clearer: enter image description here

In particular, from what I have understood so far, the attention coefficients apply to the whole semantic level (which I find not really clearly indicated on their figure)

The LHS of their figure would then be better described by this new one:

enter image description here

Hope this helps!

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ I think it would be nice if you could also explain the meaning of the expression "semantic level" in the specific context of the paper that the OP is quoting. $\endgroup$
    – nbro
    Nov 18, 2019 at 13:39
  • $\begingroup$ @nbro You're right, I read the question too fast. I took some time reading the paper. I think "semantic levels" refers to the possible interactions between the different layers of the neural network. Each layer of the neural network representing possibly a semantic level. The architecture of the RHN seems (does it?) to enable such direct interactions (cf. RHS of fig. 2 page 3 of the paper). Not too sure about this, I'm thrilled to see other's replies as I find the subject really interesting! :-) $\endgroup$
    – Jeanba
    Nov 18, 2019 at 15:01
  • $\begingroup$ You should edit your answer to include any new details you think are relevant to the question. Comments are only temporary and can be deleted later. $\endgroup$
    – nbro
    Nov 18, 2019 at 17:05
  • $\begingroup$ In the paper the authors mention that the first time they use RHN is to summarize the exogenous data into different semantics at different levels. So now, whit what @Jenba have said, I understand that the term "semantic levels" refers to the layers RHN shown below in fig. 2 at the bottom in green. Each layer is a different semantic tha is in a different level. $\endgroup$
    – MikelBa
    Nov 19, 2019 at 8:54
  • $\begingroup$ Yes I think we finally figured this out. I modified my original response with two new drawings to make it clearer. :-) $\endgroup$
    – Jeanba
    Nov 19, 2019 at 9:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .