# Which neural network can I use to solve this constrained optimisation problem?

Let $$\mathcal{S}$$ be the training data set, where each input $$u^i \in \mathcal{S}$$ has $$d$$ features.

I want to design an ANN so that the cost function below is minimized (the sum of the square of pairwise differences between model outputs) and the given constraint is satisfied, where $$w$$ is the ANN model parameter vector.

\begin{align} \min _{w}& \sum_{\{i, j\} \in \mathcal{S}}\left(f\left(w, u^{i}\right)-f\left(w, u^{j}\right)\right)^{2} \\ &f\left(w, u^{i}\right) \geq q_{\min }, \quad i \in \mathcal{S} \end{align}

What kind of ANN is suitable for this purpose?

• If the network outputs qmin constantly for every prediction, wouldn't this minimise your loss function? May 1, 2021 at 23:20
• @ Mike NZ, yes but outputing qmin constantly for every prediction is impossible. My question was what type of ANN to use. May 1, 2021 at 23:30
• @ Mike NZ, in fact what I need is to find an ANN structure which minimizes the dissimilarity of network outputs when all training data are considered. May 1, 2021 at 23:35

If I understand your query correctly, you want to create a latent space that groups similar objects. You should then probably look for Siamese networks. However, your loss function will need another term to increase dissimilarity between different labels. Otherwise, as pointed out by Mike NZ, the net would collapse(yes, it is possible). Perhaps this will give some insights.

Note that the above method is not completely unsupervised. There are, in fact, a few claims of unsupervised classifications via clustering, although your objective function would look very different. You could go through this paper(called SCAN) for more details.

Hope it helps.

If you want a (lower-dimensional)representation of objects themselves, browsing through this could help. For a complex problem, linear reductions like PCA, although helpful, aren't probably what you're looking for. Here you can try training autoencoders. The loss function would work, along with some regularization term.

• @ cybershiptrooper, the problem I want to solve is as follows: assume that we have a set S consisting of N number of people and for them we define a d-dimensional feature vector. The feature vector includes features characterizing the "quality" of people. I believe that although each person in S has different feature values, they are of similar quality. Hence, I want to design a ANN which will minimize their dissimilarity. May 2, 2021 at 12:20
• @user3489173 Oh, so you want to do something like dimensionality reduction? Try using autoencoder type architectures. I have added some more details in my answer. May 2, 2021 at 17:20
• @ cybershiptrooper It has nothing to do with dimensionality reduction. The problem is as follows. Assume that we have a set S consisting of N number of people and for them we define a d-dimensional feature vector. The feature vector includes features characterizing the "quality" of people. I believe that although each person in S has different feature values, they are of similar "high" quality. Hence, I want to design a ANN which will minimize their dissimilarity. Then, I want to use the developed model for quality quantification of a given arbitrary person. May 2, 2021 at 18:03