0
$\begingroup$

I'm currently doing a research project related to Distributed Tracing. My research has led me to a point where I think ML might be suited for our problem.

I'm looking for papers that are similar to this (even if they have other applications):

I want to match packets exiting a black-box system (outputs) to packets that enter a black box (inputs). I can do that easily in a non concurrent setting which should help me grow a training set (maybe for supervised learning), but I need an algorithm that, in a concurrent setting, can separate the different request "flows" if you will.

I hope this makes sense.

The closest thing to what I'm looking for is "Aguilera, Marcos K., et al. "Performance debugging for distributed systems of black boxes." ACM SIGOPS Operating Systems Review. Vol. 37. No. 5. ACM, 2003." but it's mostly suited for finding the dependency graph of the system, which I already know.

Thank you

$\endgroup$
0
$\begingroup$

Distributed computing is about heterogeneous computer systems which are connected over a network. This is for example a number of database servers, paired to webservers and some application servers which are sending packets back and forth. A common tool for visualizing such systems is a network simulation. This is working usually with the qemu emulator. That means, 1000s machines are started as virtual instance in the cloud with the purpose to replicate existing infrastructure. The disadvantage of qemu based simulation is, that it will consume lots of cpu processing speed. A normal desktop PC can only run 5 qemu instances at once, but not 1000s of them. That is the reason, why alternatives to network simulations are used for example blackbox modeling which goes into the direction of machine learning.

The problem with a blackbox model is, that it is working only on a semantic level. The packets are stored as full text with the aim to analyze them. But the underlying system is not simulated. On the first look, this approach saves a lot of cpu time, but it is not realistic. The better idea is to use a hybrid approach with a simplified underlying model. Instead of using qemu as a network simulator the idea is to create computersecurity serious games. A good example is PicoCTF (short of capture the flag).

Such cybersecurity games are able to simulate large amount of servers, but they are not dependent on qemu-like simulations. Instead the simulation is done within the game. For example with the PDDL syntax which defines what happens after a packet was send. It is a simulation on a higher abstraction level.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.