Which approach should I use to classify points above and below a sine function $y(x) = A + B \sin(Cx)$?

Question

In a linear regression problem, a line can divide a data set into two categories. So, basically, points above the line belong to category 1, and points below the line belong to category -1.

However, my professor has asked me to write a C++ program in which the program will classify whether the data points lie above or below a sine function.

Let me explain a bit more. So, first, we will generate a data set $$D = \{(x_i, y_i) \} \label{0}\tag{0} $$ with random $x$ and $y$ coordinates, for example, according to this equation

$$y(x) = A + B \sin(Cx)\label{1}\tag{1},$$

where $A$, $B$, and $C$ are known.

The data points above the sine function will have a label 1 on them, and the points below the function will have -1.

Now, this data set $D$ in \ref{0} has to be fed to a C++ program. This C++ program has to somehow learn the curve separating the two data point categories. After training, the program will then classify some new query data points.

The key difficulty is that the program does not know in advance that the points were scattered around a sine curve. It does not know the values of $A$, $B$, or $C$ in equation \ref{1}. It also does not know that the curve is a sine curve.

Now, this is where I am stuck. I do not know if I need to use a neural network to solve this problem. If a neural network is to be used, then I presume that backpropagation will have to be used in some way. I can generate the data set and I can feed the data into the program.

Which approach (algorithm and model) should I use to solve this problem?

I have studied linear classification with the perceptron learning algorithm, but this sine-classifier stuff is a huge step-up for me. Another important thing is that I am not allowed to use any ready-made C++ libraries for Machine Learning. If a neural network solution is needed, then I will have to design the neural network from scratch. Note that I don't need any C++ code, but I am just looking for some guidance on how to approach this problem.

Answer 1

You can try using Fourier basis functions to transform your observable variables and then apply a general linear regression model. To clarify, if you have pairs of observables $(y_i, x_i)$ where $y_i$ is $i$-th output, and $x_i$ is $i$-th input then you can transform your input variable into vector

\begin{equation} \phi = [1, \sin(x), \cos(x), \sin(2x), \cos(2x), \ldots, \sin(nx), \cos(nx)] \end{equation} where $n$ is some arbitrary number.

Then you will have a standard linear regression model \begin{equation} Y = A\Phi \end{equation} where $Y = [y_1, \ldots, y_n]$, $\Phi = [ \phi_1, \ldots, \phi_n]$ and $A$ is matrix of unknown parameters that you need to learn.