1
$\begingroup$

I have the following problem.

A bank wants to decide whether a customer can be given a loan, based on two features related to (i) the monthly salary of the customer, and (ii) his/her account balance. For simplicity, we model the two features with two binary variables $X1$, $X2$ and the class $Y$ (all of which can be either 0 or 1). $Y=1$ indicates that the customer can be given loan, and Y=0 indicates otherwise.

Consider the following dataset having four instances:

($X1 = 0$, $X2 = 0$, $Y = 0$)

($X1 = 0$, $X2 = 1$, $Y = 0$)

($X1 = 1$, $X2 = 0$, $Y = 0$)

($X1 = 1$, $X2 = 1$, $Y = 1$)

Can there be any logistic regression classifier using X1 and X2 as features, that can perfectly classify the given data?

The approach followed in the question was to calculate respective probabilities for Y=0 and Y=1 respectively. The value of $p$ obtained was $0.25$ and $(1-p)$ as $0.75$. The $\log(p/1-p)$ is coming as negative.

However, I don't understand what I need to do to understand whether there is a Logistic Regression classifier that can perfectly classify the given data.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

check it

import keras
from keras.layers import *

X = np.array([[0,0], [0,1], [1,0], [1,1]])
Y = np.array([[0], [0], [0], [1]])

input = Input(shape=(2,))
output = Dense(1, activation='sigmoid')(input)
model = keras.Model(input, output)

model.compile(keras.optimizers.Adam(1e0), 'binary_crossentropy', metrics=['acc'])
model.fit(X, Y, epochs=10, batch_size=4, verbose=1)

which produces 

Epoch 1/10
4/4 [==============================] - 0s 52ms/step - loss: 0.7503 - acc: 0.7500
Epoch 2/10
4/4 [==============================] - 0s 817us/step - loss: 0.5142 - acc: 0.7500
Epoch 3/10
4/4 [==============================] - 0s 732us/step - loss: 0.4353 - acc: 0.7500
Epoch 4/10
4/4 [==============================] - 0s 694us/step - loss: 0.3413 - acc: 1.0000
Epoch 5/10
4/4 [==============================] - 0s 633us/step - loss: 0.2817 - acc: 1.0000
Epoch 6/10
4/4 [==============================] - 0s 679us/step - loss: 0.2299 - acc: 1.0000
Epoch 7/10
4/4 [==============================] - 0s 672us/step - loss: 0.1769 - acc: 1.0000
Epoch 8/10
4/4 [==============================] - 0s 721us/step - loss: 0.1412 - acc: 1.0000
Epoch 9/10
4/4 [==============================] - 0s 694us/step - loss: 0.1193 - acc: 1.0000
Epoch 10/10
4/4 [==============================] - 0s 716us/step - loss: 0.1015 - acc: 1.0000

...so yes, you can

Also note you calculated marginal probabilities, here you want them conditioned on the input variables to actually solve for the parameters

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the explanation.Had a doubt like if we have to check for the best fit of the model for logistic regression do we have to calculate it by using the Maximum Likelihood Estimate and getting the coefficients? $\endgroup$
    – ten do
    Jun 4, 2019 at 13:51
  • $\begingroup$ so yeah you could think of it that way, given its form there is probably an analytical solution (unsure, just a guess), but for ease a simple optimizer will go a long way (especially because this is convex) $\endgroup$
    – mshlis
    Jun 4, 2019 at 13:57
  • $\begingroup$ Sure got it thanks $\endgroup$
    – ten do
    Jun 4, 2019 at 15:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .