0
$\begingroup$

Tell me why my val_acc is always the same and how to solve this problem? I saw several topics on the Internet specifically on this problem but they did not help me (for example, use SGD with different parametrs). I also tried different batch size, but alas.

Datasets:

https://yadi.sk/d/ug8IujXxWsYTxQ

Code:

from time import time
from sklearn.preprocessing import LabelEncoder
from keras.preprocessing.text import text_to_word_sequence
import re
import numpy as np
import pandas as pd
from keras.utils import to_categorical
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import Dense,LSTM,Dropout
from keras.layers import Flatten
from keras.layers.embeddings import Embedding
from keras.layers import Input, Dense, LSTM, MaxPooling1D, Conv1D
from keras.models import Model
from keras.optimizers import SGD

GLOBALDict = set()
max_length = 0

def deEmojify(string):
    emoji_pattern = re.compile("["
                           u"\U0001F600-\U0001F64F" 
                           u"\U0001F300-\U0001F5FF"  
                           u"\U0001F680-\U0001F6FF"  
                           u"\U0001F1E0-\U0001F1FF"
                           u"\U00002702-\U000027B0"
                           u"\U000024C2-\U0001F251"
                           "]+", flags=re.UNICODE)
    return emoji_pattern.sub(r'', string)


# preprocessing dataset

df = pd.read_csv('/content/lenta-ru-news-mini-mini.csv', delimiter=';')
X=[]
Y=[]

for message in df["title"]:
    cleartext = text_to_word_sequence(deEmojify(message))
    if(len(cleartext)>max_length):
        max_length = len(cleartext)

    X = X + [cleartext[0:len(cleartext)-1]]

    GLOBALDict.update(cleartext)


#print(list(GLOBALDict))  
labelencoder = LabelEncoder()
labelencoder.fit(list(GLOBALDict))

for i in range(0,len(df["title"])):
    X[i] = labelencoder.transform(X[i])


for i in range(0,len(df["title"])):
    Y = Y+[to_categorical([labelencoder.transform([text_to_word_sequence(deEmojify(df["title"][i]))[-1]])],len(GLOBALDict))[0]]

X = pad_sequences(X, maxlen=max_length, padding='post')



# define the model
model = Sequential()

model.add(Embedding(len(GLOBALDict), 32, input_length=max_length))

model.add(LSTM(8, dropout=0.2, recurrent_dropout=0.2))
#model.add(Dropout(0.2))
#model.add(Dense(128, activation='sigmoid'))
#model.add(Dropout(0.2))
#model.add(Dense(256, activation='sigmoid'))
#model.add(Dropout(0.2))
#model.add(Dense(512, activation='sigmoid'))
#model.add(Dense(1024, activation='sigmoid'))

model.add(Dense(len(GLOBALDict), activation='sigmoid'))


#sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
#model.compile(optimizer = sgd, loss='categorical_crossentropy', metrics=['accuracy'])
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
print(model.summary())

#model.fit(np.array(X), np.array(Y), epochs=5, verbose=1, validation_split=0.05, callbacks=[tensorboard])
model.fit(np.array(X), np.array(Y), epochs=5, batch_size=256, verbose=1, validation_split=0.02)

Here is the output :

Here she seems to be learning, but at the same time not learning :(

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_8 (Embedding)      (None, 304, 32)           686368    
_________________________________________________________________
lstm_8 (LSTM)                (None, 8)                 1312      
_________________________________________________________________
dense_8 (Dense)              (None, 21449)             193041    
=================================================================
Total params: 880,721
Trainable params: 880,721
Non-trainable params: 0
_________________________________________________________________
None
Train on 9805 samples, validate on 201 samples
Epoch 1/5
9805/9805 [==============================] - 24s 2ms/step - loss: 9.9652 - acc: 0.0168 - val_loss: 9.9439 - val_acc: 0.0498
Epoch 2/5
9805/9805 [==============================] - 20s 2ms/step - loss: 9.8524 - acc: 0.0200 - val_loss: 9.8172 - val_acc: 0.0498
Epoch 3/5
9805/9805 [==============================] - 21s 2ms/step - loss: 9.5719 - acc: 0.0200 - val_loss: 9.6361 - val_acc: 0.0498
Epoch 4/5
9805/9805 [==============================] - 21s 2ms/step - loss: 9.2304 - acc: 0.0200 - val_loss: 9.5272 - val_acc: 0.0498
Epoch 5/5
9805/9805 [==============================] - 21s 2ms/step - loss: 9.0019 - acc: 0.0200 - val_loss: 9.4856 - val_acc: 0.0498
$\endgroup$
  • $\begingroup$ Welcome to SE:AI. This may not be the best forum for this question, unless you are inquiring about the concept in general, as opposed to a specific solution. You may want to have a look at Data Science. (Let me know if you'd like me to migrate the question.) $\endgroup$ – DukeZhou Aug 13 at 21:38
0
$\begingroup$

I'm pretty sure accuracy is the number or percentage of outputs that exactly matches the target, which means probably one or a few of the nodes, by chance has got -1 or 1 which happened to be the target. It's no worry though, as long as the loss is changing, your nn is learning :)

$\endgroup$
  • $\begingroup$ hmm, well, I'll try to continue to teach $\endgroup$ – alex-rudenkiy Aug 13 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.