Use Machine Learning/Artificial Intelligence to predict next number (n+1) in a given sequence of random increasing integers

Question

The AI must predict the next number in a given sequence of incremental integers (with no obvious pattern) using Python but so far I don't get the intended result! I tried changing the learning rate and iterations but so far no luck!

Example sequence: [1, 3, 7, 8, 21, 49, 76, 224]

Expected result: 467

Result found : 2,795.5

Cost: 504579.43

This is what I've done so far:

import numpy as np

# Init sequence
data =\
    [
        [0, 1.0], [1, 3.0], [2, 7.0], [3, 8.0],
        [4, 21.0], [5, 49.0], [6, 76.0], [7, 224.0]
    ]

X = np.matrix(data)[:, 0]
y = np.matrix(data)[:, 1]

def J(X, y, theta):
    theta = np.matrix(theta).T
    m = len(y)
    predictions = X * theta
    sqError = np.power((predictions-y), [2])
    return 1/(2*m) * sum(sqError)

dataX = np.matrix(data)[:, 0:1]
X = np.ones((len(dataX), 2))
X[:, 1:] = dataX

# gradient descent function
def gradient(X, y, alpha, theta, iters):
    J_history = np.zeros(iters)
    m = len(y)
    theta = np.matrix(theta).T
    for i in range(iters):
        h0 = X * theta
        delta = (1 / m) * (X.T * h0 - X.T * y)
        theta = theta - alpha * delta
        J_history[i] = J(X, y, theta.T)
     return J_history, theta
print('\n'+40*'=')

# Theta initialization
theta = np.matrix([np.random.random(), np.random.random()])

# Learning rate
alpha = 0.02

# Iterations
iters = 1000000

print('\n== Model summary ==\nLearning rate: {}\nIterations: {}\nInitial 
theta: {}\nInitial J: {:.2f}\n'
  .format(alpha, iters, theta, J(X, y, theta).item()))
print('Training model... ')

# Train model and find optimal Theta value
J_history, theta_min = gradient(X, y, alpha, theta, iters)
print('Done, Model is trained')
print('\nModelled prediction function is:\ny = {:.2f} * x + {:.2f}'
  .format(theta_min[1].item(), theta_min[0].item()))
print('Cost is: {:.2f}'.format(J(X, y, theta_min.T).item()))

# Calculate the predicted profit
def predict(pop):
    return [1, pop] * theta_min

# Now
p = len(data)
print('\n'+40*'=')
print('Initial sequence was:\n', *np.array(data)[:, 1])
print('\nNext numbers should be: {:,.1f}'
  .format(predict(p).item()))

UPDATE Another method I tried but still giving wrong results

import numpy as np
from sklearn import datasets, linear_model

# Define the problem
problem = [1, 3, 7, 8, 21, 49, 76, 224]

# create x and y for the problem

x = []
y = []

for (xi, yi) in enumerate(problem):
    x.append([xi])
    y.append(yi)

x = np.array(x)
y = np.array(y)
# Create linear regression object
regr = linear_model.LinearRegression()
regr.fit(x, y)

# create the testing set
x_test = [[i] for i in range(len(x), 3 + len(x))]

# The coefficients
print('Coefficients: \n', regr.coef_)
# The mean squared error
print("Mean squared error: %.2f" % np.mean((regr.predict(x) - y) ** 2))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % regr.score(x, y))

# Do predictions
y_predicted = regr.predict(x_test)

print("Next few numbers in the series are")
for pred in y_predicted:
    print(pred)

Answer 1

I think your code works fine for what is meant to be doing - fitting a linear regression model. The problem here is that you are using a linear model. Linear model does not have an adequate approximation capacity, it will only be able to fit data that is described by a linear function. Here, you gave a random sequence of numbers, that is very difficult for linear model to approximate. I would advise you to try 2 things:

1) Try something simpler first. Instead of doing a random sequence of numbers do a linear sequence of numbers for example a function like $y = 2x$ or maybe affine function like $y = 2x + 5$. So you would have a sequence like:

$2, 4, 6, 8 ...$ or $7, 9, 11, 13, ...$

If you manage to get that working try a nonlinear function like $x^2$ for example.

2) Instead of using a linear model try a nonlinear model. For example a polynomial regression model. Especially powerful function approximators are neural networks. In theory, a neural network with single hidden layer can approximate an arbitrary continuous function under some conditions (Universal approximation theorem) , so you could try to see how would a neural network solve the problem, there are several open source neural network libraries that you could try.

Answer 2

Like others said you can't approximate this with a linear regression model.

A PRM that approximates a solution could give you the following:
$y = 0.948 + x + 0.00085*x^6$ ~
$y = 237/250 + x + (17/20000)*x^6$
For $x = 9$, $y \simeq 462$

or

$y = 0.9258 + x + 0.00086*x^6$ For $x = 9$, $y \simeq 466.965$

UPDATE
An approximation of course, may be in the range of:
$y = 2^{(x + 1)} - 2^x$ -the model you propose-
Goodness of fit: 0.968475 and Mean Square Error = 685.111

Based on this range a better approximation would be:
$y = 2^x + (-1/2)*x^2$
with $R^2$
Goodness of fit = 0.995
Mean Square Error: 89.0278