# Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch):

I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $$f$$ seeks to learn a new network $$g$$ whose last few layers behave very similarly to $$f$$. Thus, the "knowledge of f" is encoded into the output layers of $$g$$, but only approximately.

This is essentially what is described in the "fine-tuning" section of this Keras page. However, I'm looking for an academic reference..

Question Does anyone know of some article which use this kind of setup?

Formal Question:

Suppose that we have a feed-forward network $$f(x):=f_n\circ \dots \circ f_1(x) \mbox{ where } f_i(x)=\sigma(A_ix +b_i),\, f_n(x)=A_n x+ b_n ,$$ for $$i=1,\dots,n-1$$, trained on some given learning task. Given some $$\epsilon>0$$, and some loss functions $$L_1,L_2$$, I'm interested in transferring the "knowledge" of the network $$f$$ to a different learning task by training a new network $$g(x):=g_N\circ \dots \circ g_1(x) \mbox{ where } g_i(x)=\sigma(A_i'x +b_i'),\, g_n(x)=A_n' x+ b_n; ,$$ for $$i=1,\dots,N-1$$; by optimizing the following \begin{aligned} \min & \, L_2(g) \\ \mbox{ such that: } & L_1(f,g_N\circ\dots \circ g_{N-2} ) <\epsilon \end{aligned}

Related Post Here.