In the time since I asked this question, I have been able to combine Tensorflow and Chainer considerably well. That being said, one should try to avoid combining deep learning frameworks if one can for a few reasons:
- It doubles the amount of documentation one needs to reference
- It makes it difficult for new developers to become familiar with the
- It creates multiple data transfer bottlenecks when one
has to transfer data between the deep learning different
configurations such as: CPU -> GPU (Tensorflow) -> CPU -> GPU
(Chainer) -> CPU ... etc.
That being said, I combined them because of time constraints on a research project which would not be finding its way into an industrial product. In this research setting, I still wouldn't recommend combining frameworks if one can avoid it but if it means doubling the time it takes to do the project for a proof of concept or experiment, then it seems reasonable and can drastically speed up development until the missing functionality is added to the desired framework.
There are 2 issues with integration:
Abstracting the networks so that the main
training/testing script does not depend on the framework.
This is important in the event that the missing functionality of your desired framework is added. It also helps new developers understand what the overall algorithm or experiment is without being bogged down by framework details.
Tensorflow and similar frameworks which require a Session object.
Chainer, PyTorch (which came from Chainer), and any other framework that doesn't require a something like Tensorflow's Session all play very nicely together and can be used (almost) interchangeably. For Session based frameworks, the main script/loop/experiment must all take place within a Session. With proper abstraction, my experiment only required a
with tf.Session() as sess: to be used in the main experiment and was the only reference that Tensorflow was being used.
Although I still don't recommend combining frameworks for the reasons specified above, if one abstracts their framework calls enough, it is almost seamless to integrate deep learning frameworks.