4
$\begingroup$

The AI must predict the next number in a given sequence of incremental integers (with no obvious pattern) using Python but so far I don't get the intended result! I tried changing the learning rate and iterations but so far no luck!

Example sequence: [1, 3, 7, 8, 21, 49, 76, 224]

Expected result: 467

Result found : 2,795.5

Cost: 504579.43

This is what I've done so far:

import numpy as np

# Init sequence
data =\
    [
        [0, 1.0], [1, 3.0], [2, 7.0], [3, 8.0],
        [4, 21.0], [5, 49.0], [6, 76.0], [7, 224.0]
    ]

X = np.matrix(data)[:, 0]
y = np.matrix(data)[:, 1]

def J(X, y, theta):
    theta = np.matrix(theta).T
    m = len(y)
    predictions = X * theta
    sqError = np.power((predictions-y), [2])
    return 1/(2*m) * sum(sqError)

dataX = np.matrix(data)[:, 0:1]
X = np.ones((len(dataX), 2))
X[:, 1:] = dataX

# gradient descent function
def gradient(X, y, alpha, theta, iters):
    J_history = np.zeros(iters)
    m = len(y)
    theta = np.matrix(theta).T
    for i in range(iters):
        h0 = X * theta
        delta = (1 / m) * (X.T * h0 - X.T * y)
        theta = theta - alpha * delta
        J_history[i] = J(X, y, theta.T)
     return J_history, theta
print('\n'+40*'=')

# Theta initialization
theta = np.matrix([np.random.random(), np.random.random()])

# Learning rate
alpha = 0.02

# Iterations
iters = 1000000

print('\n== Model summary ==\nLearning rate: {}\nIterations: {}\nInitial 
theta: {}\nInitial J: {:.2f}\n'
  .format(alpha, iters, theta, J(X, y, theta).item()))
print('Training model... ')

# Train model and find optimal Theta value
J_history, theta_min = gradient(X, y, alpha, theta, iters)
print('Done, Model is trained')
print('\nModelled prediction function is:\ny = {:.2f} * x + {:.2f}'
  .format(theta_min[1].item(), theta_min[0].item()))
print('Cost is: {:.2f}'.format(J(X, y, theta_min.T).item()))

# Calculate the predicted profit
def predict(pop):
    return [1, pop] * theta_min

# Now
p = len(data)
print('\n'+40*'=')
print('Initial sequence was:\n', *np.array(data)[:, 1])
print('\nNext numbers should be: {:,.1f}'
  .format(predict(p).item()))

UPDATE Another method I tried but still giving wrong results

import numpy as np
from sklearn import datasets, linear_model

# Define the problem
problem = [1, 3, 7, 8, 21, 49, 76, 224]

# create x and y for the problem

x = []
y = []

for (xi, yi) in enumerate(problem):
    x.append([xi])
    y.append(yi)

x = np.array(x)
y = np.array(y)
# Create linear regression object
regr = linear_model.LinearRegression()
regr.fit(x, y)

# create the testing set
x_test = [[i] for i in range(len(x), 3 + len(x))]

# The coefficients
print('Coefficients: \n', regr.coef_)
# The mean squared error
print("Mean squared error: %.2f" % np.mean((regr.predict(x) - y) ** 2))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % regr.score(x, y))

# Do predictions
y_predicted = regr.predict(x_test)

print("Next few numbers in the series are")
for pred in y_predicted:
    print(pred)
$\endgroup$
  • 2
    $\begingroup$ Hi! Please, ask this question on Data Science SE or Stack Overflow. $\endgroup$ – nbro Apr 27 at 16:20
  • 1
    $\begingroup$ If there is no underlying pattern and the sequence is unbounded a simple NN cannot approximate it. $\endgroup$ – DuttaA Apr 27 at 17:59
  • $\begingroup$ Cross-posted here: datascience.stackexchange.com/q/51033/924 $\endgroup$ – Anony-Mousse Apr 28 at 9:54
  • $\begingroup$ are you hunting my posts? @Anony-Mousse, I posted the same thread on multiple Stack platforms $\endgroup$ – Thorvald Ólavsen V. Apr 28 at 11:15
  • 1
    $\begingroup$ @nbro please be clear to suggest to only ask in one place, and not to duplicate questions. Otherwise future users will have to look for answers in many places. For the rationale of not allowing crossposting see meta.stackexchange.com/questions/64068/… Thank you. $\endgroup$ – Anony-Mousse Apr 29 at 1:31
3
$\begingroup$

I think your code works fine for what is meant to be doing - fitting a linear regression model. The problem here is that you are using a linear model. Linear model does not have an adequate approximation capacity, it will only be able to fit data that is described by a linear function. Here, you gave a random sequence of numbers, that is very difficult for linear model to approximate. I would advise you to try 2 things:

1) Try something simpler first. Instead of doing a random sequence of numbers do a linear sequence of numbers for example a function like $y = 2x$ or maybe affine function like $y = 2x + 5$. So you would have a sequence like:

$2, 4, 6, 8 ...$ or $7, 9, 11, 13, ...$

If you manage to get that working try a nonlinear function like $x^2$ for example.

2) Instead of using a linear model try a nonlinear model. For example a polynomial regression model. Especially powerful function approximators are neural networks. In theory, a neural network with single hidden layer can approximate an arbitrary continuous function under some conditions (Universal approximation theorem) , so you could try to see how would a neural network solve the problem, there are several open source neural network libraries that you could try.

$\endgroup$
  • $\begingroup$ well I found a pattern and it goes like this: we take the first element in the array and then we take it's index and next index and then the AI makes the decision which number to choose in that interval, that's how we got the number for example we got 3(2nd number in array) by applying the same logic like this [2^1(index of 2nd element), **3**(number 3 exists somewhere between this interval), 2^2(next index)] and so on and so forth for each element (his index and the next...) but since I have no previous knowledge in AI and machine learning I have no idea how to apply this $\endgroup$ – Thorvald Ólavsen V. May 5 at 11:46
2
$\begingroup$

Like others said you can't approximate this with a linear regression model.

A PRM that approximates a solution could give you the following:
$y = 0.948 + x + 0.00085*x^6$ ~
$y = 237/250 + x + (17/20000)*x^6$
For $x = 9$, $y \simeq 462$

or

$y = 0.9258 + x + 0.00086*x^6$ For $x = 9$, $y \simeq 466.965$

UPDATE
An approximation of course, may be in the range of:
$y = 2^{(x + 1)} - 2^x$ -the model you propose-
Goodness of fit: 0.968475 and Mean Square Error = 685.111

Based on this range a better approximation would be:
$y = 2^x + (-1/2)*x^2$
with $R^2$
Goodness of fit = 0.995
Mean Square Error: 89.0278

$\endgroup$
  • $\begingroup$ I actually found a pattern it is not random the pattern goes like this: you take the element's current index and it's next index and you apply this interval : [2^index, 2^next index] for example for the number 7 in the given example above we do this [2^2, 2^3] the number 7 is somewhere between that interval, the AI must predict based on this pattern. if you have any solution proposed please be kind and guide me through! $\endgroup$ – Thorvald Ólavsen V. May 9 at 10:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.