# Tag Info

4

The mean squared error (MSE), $J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the data that are not appropriate for classification. Though, as an optimization objective, it is still possible to attempt to minimize MSE even in a classification ...

4

The most likely explanation is that the neural network model proposed has a much higher model capacity than the logistic regression model. In fact, the neural network used in the code has 246,343 trainable parameters; the full Iris data set has only 150 samples and only four features – so the model is much more complex than the training data. A neural ...

3

Try Rectification Improve the features available to your model, Remove some of the NOISE present in the data. In audio data, a common way to do this is to smooth the data and then rectify it so that the total amount of sound energy over time is more distinguishable. # Rectify the audio signal audio_rectified = audio.apply(np.abs) You can also calculate ...

2

Nope! Our number of coefficients will be driven by the vocabulary, and we'll use each of those 10K samples to estimate values for those coefficients - so, 'just' 100K samples. However, word frequency in human languages follows a Zipf distribution => most of those words will be rare, seen in only a few samples (=> won't even be able to determine whether ...

2

I see why you might be confused. First, the logistic-loss or log-loss is technically called cross-entropy loss. This function is very simple: $CE = -[y \log(p) + (1 - y) \log(1 - p)]$ This tells basically if the predicted class $y$ was right $y=1$ then the loss is $CE=-\log(p)$, if the predicted class was not the right one then the loss is $CE=-\log(1-p)$. ...

2

The straight dashed-line shows the typical decision line in logistic regression or any linear classifier. The dashed-circle shows the decision line from SVM. Obviously, since the data is not linearly separable in the original 2D feature space, if someone makes a higher dimension space by taking into account non-linear interaction of the original 2 features ...

1

Yes, it absolutely makes sense as the Iris dataset is linearly separable (in the sense that linear decision boundaries are near optimal). This can just about be seen in the scatter plots: Particularly look at petal width versus petal length, two linear decision boundaries for that pair of variables gives a very low error rate already. This shouldn't be ...

1

Just realised the issue - super subtle (at least for a Python novice like me) - I implemented numerical gradient checking (as I should have done from the start) and saw the gradient descent was working incorrectly. The dataset Y was created as a datatype "uint8". In the line of the back-propagation where I calculate the derivative with respect to &...

1

In generalised Linear models, each output variable $y_i$ is modelled as a distribution from the exponential family, with the hypothesis function $h_{\theta}(x)$ for a given $\theta$ as the expected value of $y_i$ and maximum likelihood estimation is usually the method used to solve GLM's.

1

I think i found out how that works, so i made a short article about it . https://medium.com/@kourloskostas/python-spam-filter-86b21d7d1564 I hope it helps!

1

I mean you technically could (it's not going to break or something) however, cross entropy is much better suited for classification as it penalizes for misclassification errors: have a look at the function: when you are wrong the loss goes to infinity: you are either from one class or another. MSE is designed for regression where you have nuance: you get ...

1

check it import keras from keras.layers import * X = np.array([[0,0], [0,1], [1,0], [1,1]]) Y = np.array([, , , ]) input = Input(shape=(2,)) output = Dense(1, activation='sigmoid')(input) model = keras.Model(input, output) model.compile(keras.optimizers.Adam(1e0), 'binary_crossentropy', metrics=['acc']) model.fit(X, Y, epochs=10, batch_size=4,...

1

I refer you to Prof. Tom Mitchell's (the lecturer in the video) draft chapter as the best explanation I could find. I will try to explain it in layman's terms here. Given a boolean problem, our logistic regression classifier will assign the label $Y=0$ iff $w_0 + \sum_{i=1}^{n}w_{i}X_{i} > 0$ (from the video you posted; eqn. 18 in the draft chapter). ...

Only top voted, non community-wiki answers of a minimum length are eligible