# Tag Info

13

There are many approaches to this kind of problem. The most obvious one is to create new features. The best features I can come up with is to transform the coordinates to spherical coordinates. I have not found a way to do it in playground, so I just created a few features that should help with this (sin features). After 500 iterations it will saturate and ...

9

Any supervised learning (SL) problem can be cast as an equivalent reinforcement learning (RL) one. Suppose you have the training dataset $\mathcal{D} = \{ (x_i, y_i \}_{i=1}^N$, where $x_i$ is an observation and $y_i$ the corresponding label. Then let $x_i$ be a state and let $f(x_i) = \hat{y}_i$, where $f$ is your (current) model, be an action. So, the ...

9

Ideally neural networks should be able to find out the function out on it's own without us providing the spherical features. After some experimentation I was able to reach a configuration where we do not need anything except $X_1$ and $X_2$. This net converged after about 1500 epochs which is quite long. So the best way might still be to add additional ...

6

Yes. For instance, the popular softmax regression gives you probability distribution for each class. Yes. Softmax is a regression over a set of discrete classes. We can use regression for classification, the most common strategy is to grab the most likely class for the prediction.

4

By cheating... theta is $\arctan(y,x)$, $r$ is $\sqrt{(x^2 + y^2)}$. In theory, $x^2$ and $y^2$ should work, but, in practice, they somehow failed, even though, occasionally, it works.

4

I think it depends on you application and what data you have available. If the prediction of body temperature itself doesn't have to be accurate and classes like COLD, NORMAL, and HOT will suffice, you should stay with a classification. There isn't a cut off but as you increase the number of classes that represent numbers on the same scale, it may become ...

4

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its convex, and all-around easy. but for alot of larger problems, this wont work. As you said you want the machine to do the work, so this is (kinda) where deeper ...

4

If your model is gradient-based, such as a neural network, then may also be able to use gradient methods to drive virtual inputs: Freeze all network weights to the trained version Define a loss function that decribes how you want the output - or any internal measure - to behave. E.g. to maximise the output, the loss function can simply be the negative of ...

3

Of course, it's possible to define a problem where there is no relationship between input $x$ and output $y$. In general, if the mutual information between $x$ and $y$ is zero (i.e. $x$ and $y$ are statistically independent) then the best prediction you can do is independent of $x$. The task of machine learning is to learn a distribution $q(y|x)$ that is as ...

3

In reality GANs are not made for image classification, but for data generation, and they have gained popularity on image generation. They are also used for tabular data generation, see for example TGAN, or for time series generation, e.g. Quant GAN. You have even some application for the field of graphs and networking, e.g. NetGAN and GraphGAN.

3

The main distinction between tasks is 'classification' vs 'regression'. In classification you would try to identify the presence of a cloud or not in an image, if you want to predict the level of 'cloudness' with continuous values you are then performing a regression task. I'm not aware about state-of-the models specific for images, but you can potentially ...

3

In regression, the goal is to approximate a function $f: \mathcal{I} \rightarrow \mathbb{R}$, so $f(x) \in \mathbb{R}$. In other words, in regression, you want to learn a function whose outputs can be any number, so not necessarily just a number in the range $[0, 1]$. You use the sigmoid as the activation function of the output layer of a neural network, for ...

3

Try Rectification Improve the features available to your model, Remove some of the NOISE present in the data. In audio data, a common way to do this is to smooth the data and then rectify it so that the total amount of sound energy over time is more distinguishable. # Rectify the audio signal audio_rectified = audio.apply(np.abs) You can also calculate ...

2

See the blog post Why You Should Use Cross-Entropy Error Instead Of Classification Error Or Mean Squared Error For Neural Network Classifier Training (2013) by James D. McCaffrey. It should give you an intuition of why the average cross-entropy (ACE) is more appropriate than MSE (but MSE is also applicable). In a few words, $\tanh$ + MSE is like sigmoid + ...

2

Since you're termed the problem you're trying to solve is to "measure the body temperature of a person", the output should be a continuous valued number. If the problem statement had been to rank or classify a person's body temperature as fever/healthy, then this could be a classification problem. Deep learning employ neuron units at the output layer ...

2

You're probably looking for regression, either linear or non-linear, which usually refers to a set of methods that can be used to predict a continuous (or numerical) value (the value of the so-called dependent variable), given one or more possibly numerical values (the values of the independent variables). (The other common task is called classification, ...

2

This should be possible given the fact that ANNs have the ability to do the feature engineering and feature selection tasks by themselves. This means that given a lesser number of input parameters, the model will be able to generate and select additional features by itself. You will obviously not be able to understand or model these features manually. The ...

2

You can not use error to reliably measure accuracy. Error is best used as a measure of how fast the model is currently learning. As an example, using different loss functions (cross entorpy vs MSE) results in massively different values for the error at similar accuracy. Also considering this, an error of 0.0000000001 quite often has lower validation set ...

2

I report three definitions of machine learning (ML) and I also explain that ML can be divided into multiple sub-tasks or sub-categories in this answer. However, it may not always be clear why classification, regression, or clustering can be considered machine learning tasks or can be solved with ML algorithms/programs, so let me explain why these tasks can ...

2

It isn't too surprising to see behaviour like this, since you're using $\mathrm{ReLU}$ activation. Here is a simple result which explains the phenomenon for a single-layer neural network. I don't have much time so I haven't checked whether this would extend reasonably to multiple layers; I believe it probably will. Proposition. In a single-layer neural ...

1

I'm not aware of a direct way for finding the best NN architecture for a given task, but the recommended way, as far as I know, is to devise a network that can overfit the training data, and then apply regularization on top of it. That way, you can be almost sure you're not underfitting/underperforming due to network capacity.

1

Just for clarification: your description (1 sample per minute) does not match the example data (far fewer data points which is understandable, but also two data points in one minute which contradicts the initial assertion.) If your actual measurements are like that you should first work on the sampling process to get reliable data. For creating predictions, ...

1

The quick answer is that you want to use an activation function on the output layer that does not compress values to $(0,1)$. Depending on your software, this might be called "linear" or "identity". It looks like Keras just wants you to leave off the activation function: model.add(Dense(1)). The typical way of thinking of a neural network as a classifier (...

1

Lets mock some data up. "100 numbers, each one is a parameter, they together define a number X(also given)" # i.e. size of X_train -> [n x d] # i.e. size of X_train -> [??? x 100] , when d = 100 # "I have 20000 instances for training" # i.e. size of X_train -> [20000 x 100], when n = 20000 import torch import numpy as np X_train = torch.rand(...

1

Some ideas out the top of my head: In the case of $dy/dx_2>0$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds In the case of $y + x_5 + x_7 < K$ you could use a clipping function on the output layer?

1

Using two value and using MSE is probably a better approach. I'd you combine the value to one value, like the case of summation, the network may fits to output 0 on one axis and the value on the other. The method you propose also have the same issue. There are many combination to the real distance, but only one is correct. For a neural network to learn ...

1

TL;DR:One does not know ahead of time what hyper-parameters will achieve optimal performance. So what you need is an iterative implementation strategy: Implementation Strategy When working with neural networks it is key to make sure that you spend your time wisely. It is possible to spend lots of time on a dead end simply because you made an assumption ...

1

To know the form of your non-linear function, firstly you should define the type of problem you are dealing with such as an image classification task. Secondly, pick the activation functions based on your task such as sigmoid, Tanh, ReLu, LeadyRelu, Softmax etc. Overall, your ANN performance mainly depends on the number of hidden layers (hidden units), ...

1

Just as a general remark, notice that technically we don't use the term "accuracy" for regression settings, such as yours - only for classification ones. If RMSE is 'in the units of the quantity being estimated', does this mean we can say: "The network is on average (1-SQRT(0.019))*100 = 86.2% accurate"? No. The advantage of the RMSE, as you have ...

Only top voted, non community-wiki answers of a minimum length are eligible