Hey I am training an initialized Neural Network with this Method

public void rlearn(ArrayList<Tuple> tupels, double learningrate, double discountfactor) {

    MLDataSet set = new BasicMLDataSet();
    MLDataSet input = new BasicMLDataSet();
    MLDataSet ideal = new BasicMLDataSet();
    for(int i = 0; i > tupels.size()-1; i++) {
        MLData datain = new BasicMLData(45);
        MLData dataout = new BasicMLData(4);
        int index = 0;
        for(double w : tupels.get(i).statefirst.elements) {
        } //added State
        //Add new Q Values
        index = 0;
        for(int k = 0; k < tupels.get(i).qactions.elements.length;k++) {

            if(k == tupels.get(i).actionTaken) {
                //New Q - Value
                double currentQValue = tupels.get(i).qactions.getElement(k);
                double reward = tupels.get(i).rewardafter;
                //Calculate maximal Q Value of next State
                double max = Double.MIN_VALUE;
                for(double w : tupels.get(i+1).qactions.elements) {
                    if(w > max) {
                        max = w;
                dataout.add(index++,currentQValue + learningrate*(reward + discountfactor*max - currentQValue));
            } else {
                dataout.add(index++, tupels.get(i).qactions.getElement(k));

    System.out.println("Training Data: " + set.size());
    if(set.size() != 0) {
    Backpropagation prop = new Backpropagation(nn, set);

    System.out.println("Training Done: " + prop.getError());

Unfortunately this does not work pretty well. The Error is converging to Zero (pretty fast from 10000), but the Neural Net does not seem to have learned something (it is big enough)

The actual goal is to create a NN which can play something similar like astroids. Therefore the goal is to survive as long as possible. After every frame the NN gets +1 Point, if it dies -100;

Edit: The game looks like https://youtu.be/qxGR2bgj8VY (I coded it) but the AI can only go up,down,left and right. Furthermore, the AI can only steer the Player every 1/3 second...


The success probability of machine learning depends entirely from the domain. An “xor problem” gets learned with most neural networks great, image recognition is somewhere in the middle and training networks for playing video games is hard. Hard means, that the described situation with the missing learning progress isn't an exception but it's normal.

Even without knowing the exact asteroids game it's sure, that the game is too complicated. That means, the player has too much options, the number of asteroids is too high, and the control has too much degree of freedoms. Solving the problem is easy: 1. try an easier to solve game, in which the robot has to minimize a distance to the goal and the distance is short. 2. use a different kind of learning technique. A good one is a hybrid of fuzzy control and neural networks.

  • $\begingroup$ Thanks for your suggestion! I gave the AI only 4 degrees of freedom. The game looks like this youtu.be/qxGR2bgj8VY, but the AI can only go up, down, left and right. $\endgroup$ – TVSuchty Jan 3 at 9:16
  • $\begingroup$ Also it uses the raw data. Should I normalize it? The AI can only steer 3 times per second... $\endgroup$ – TVSuchty Jan 3 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.