1
$\begingroup$

A bank wants to decide whether a customer can be given a loan, based on two features related to (i) the monthly salary of the customer, and (ii) his/her account balance. For simplicity, we model the two features with two binary variables $X1$, $X2$ and the class $Y$ (all of which can be either 0 or 1). $Y=1$ indicates that the customer can be given loan, and Y=0 indicates otherwise. Consider the following dataset having four instances: ($X1 = 0$, $X2 = 0$, $Y = 0$) ($X1 = 0$, $X2 = 1$, $Y = 0$) ($X1 = 1$, $X2 = 0$, $Y = 0$) ($X1 = 1$, $X2 = 1$, $Y = 1$)
Can there be any logistic regression classifier using X1 and X2 as features, that can perfectly classify the given data?

The approach followed in the question was to calculate respective probabilities for Y=0 and Y=1 respectively. The value of $p$ obtained was $0.25$ and $(1-p)$ as $0.75$.The $log(p/1-p)$ is coming as negative. But I am stuck at this place as to which parameter to be calculated which decides that logistic regression would classify/not classify the data ?

$\endgroup$
1
$\begingroup$

check it

import keras
from keras.layers import *

X = np.array([[0,0], [0,1], [1,0], [1,1]])
Y = np.array([[0], [0], [0], [1]])

input = Input(shape=(2,))
output = Dense(1, activation='sigmoid')(input)
model = keras.Model(input, output)

model.compile(keras.optimizers.Adam(1e0), 'binary_crossentropy', metrics=['acc'])
model.fit(X, Y, epochs=10, batch_size=4, verbose=1)

which produces

Epoch 1/10
4/4 [==============================] - 0s 52ms/step - loss: 0.7503 - acc: 0.7500
Epoch 2/10
4/4 [==============================] - 0s 817us/step - loss: 0.5142 - acc: 0.7500
Epoch 3/10
4/4 [==============================] - 0s 732us/step - loss: 0.4353 - acc: 0.7500
Epoch 4/10
4/4 [==============================] - 0s 694us/step - loss: 0.3413 - acc: 1.0000
Epoch 5/10
4/4 [==============================] - 0s 633us/step - loss: 0.2817 - acc: 1.0000
Epoch 6/10
4/4 [==============================] - 0s 679us/step - loss: 0.2299 - acc: 1.0000
Epoch 7/10
4/4 [==============================] - 0s 672us/step - loss: 0.1769 - acc: 1.0000
Epoch 8/10
4/4 [==============================] - 0s 721us/step - loss: 0.1412 - acc: 1.0000
Epoch 9/10
4/4 [==============================] - 0s 694us/step - loss: 0.1193 - acc: 1.0000
Epoch 10/10
4/4 [==============================] - 0s 716us/step - loss: 0.1015 - acc: 1.0000

...so yes, you can

Also note you calculated marginal probabilities, here you want them conditioned on the input variables to actually solve for the parameters

$\endgroup$
  • $\begingroup$ Thanks for the explanation.Had a doubt like if we have to check for the best fit of the model for logistic regression do we have to calculate it by using the Maximum Likelihood Estimate and getting the coefficients? $\endgroup$ – ten do Jun 4 at 13:51
  • $\begingroup$ so yeah you could think of it that way, given its form there is probably an analytical solution (unsure, just a guess), but for ease a simple optimizer will go a long way (especially because this is convex) $\endgroup$ – mshlis Jun 4 at 13:57
  • $\begingroup$ Sure got it thanks $\endgroup$ – ten do Jun 4 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.