2
How about dividing the problem?
You can first train a classification model that predicts the type of function (linear or exponential).
Then you can use your seperately trained nn depending on the classification output.
P.S.
I'm not sure why you would use a neural network for this problem. Fitting a linear/exponential function seems to be a relatively simple ...
2
Transformer models have limited sequence length at inference time
because of positional embeddings. But there are workarounds.
Self-attention in transformer does not distinguish the order of keys/values,
it works as if the sequence is a bag of words.
So to expose the sequence order to the model, one typically adds an extra
"positional embedding" ...
2
What you refer to as logic AI is a subset of what is called symbolic AI, as you manipulate symbols, according to certain rules (which could be rules of logic). These rules are either authored by a human being, or they can be learned from examples. There are algorithms to derive decision trees (eg ID3) or other rule sets from training data. But the important ...
1
I would say that in general situation more estimators are better.
RandomForest fits a lot of estimators - decision trees that take a subset of data (obtained sampling with replacement) and subset of features (by default sqrt(n_features) in sklearn).
Each of these estimators is noisy and prone to overfitting, producing a complicated decision surface.
But when ...
1
Is this a valid implementation of second-order regression?
No, but it is not far off.
To perform a full second-order regression, you will need all terms for $x_{i,j}x_{i,k}$ where the first index is the example and the second index the feature. This includes every combination of two input variables. In your element-wise squaring you only produce terms $x_{i,...
1
Intuitively, I feel like if there are 30 foods, each with 2 states, then that is 60 states, no $2^{30}$.
Let's try it with 3 pellets. If you are right there would be $2 \times 3 = 6$ states, if the authors are right there would be $2^3 = 8$ states.
Using * for a pellet, and - for a space, we have the following states:
* * *
* * -
* - *
* - -
- * *
- * -
- -...
1
For deep learning models, embedding vectors have become the standard way of encoding text features almost immediately after their introduction.
The reason for this is that neural networks work with data encoded with continuous values ranging from 0 to 1 (or sometimes from -1 to 1). Bag of Words and TF-IDF can be modified to produce values in this range, but ...
1
Here is a quick idea: first calculate the count of how many times each word occurs in these documents (I don't know whether to lowercase them or not, do interface and Interface mean different things?), and sort them in the descending order of occurrence. Most frequent words can be called "keywords" of your configurations (such as vlan), or maybe ...
1
I have found the answer to this doubt I had. Here, 0.007709330413490534 = 1 / S, q = input, Z = 3. Basically, this is the formula to quantize the input value. If you pull out 1/S then it becomes clear. Here, is an article related to this topic.
1
As DKDK said, Indeed one could fit both linear and exponential function and see which one has smaller residual, without using any complex AI.
But OTOH this could be a great toy-problem for learning about neural networks. You could have a network with these parts:
A network with a final sigmoid activation, which predicts whether the function is linear or not....
1
I will give you a few scenarios where matrix factorisation stills works pretty well.
Topic Modelling : Given a matrix of Document as Rows and Terms/Words as column you can use Non Negative Matrix factorisation to identify Topics. Number of Topics is defined by user or can be treated as hyperparameter.
Image Ref : https://towardsdatascience.com/nmf-a-visual-...
1
This is the case as the loss doesn't have to monotonically decrease when it's updated in the negative direction.
For example:
Let $L(\theta) = \theta^2 $ and $\theta_0= 3$
Let the subscript n in $\theta_n$ denote the iteration number.
Then $\nabla_{\theta}L(\theta_0) = 2*\theta = 2*3 = 6$
For the loss to decrease in this case $\epsilon < 1$ needs to hold ...
Only top voted, non community-wiki answers of a minimum length are eligible