So, I have this huge amount of data, which has 7 vector features (float from 0 to 1). I am trying to build a kind of recommendation system, with a twist (it uses agents and negotiations and narratives; narratives meaning, that there will be temporal and partial order causal link dynamics, or "short term memory").
There is network effects in the data, since the more agents I have, the more potential there will be for match making; in simulation phase I will be just using randomness, instead of real data, but I believe the real dataset will also be more or less random: of course I think there will be nice clusters of correlations for the vectors, but I also want to understand the more general aspects of the problem, instead of just this specific use-case.
The agents will be generating a list of artifacts in co-operation and also evaluate how the generated list of artifacts match their preferences. The agents will be recommending items from their own list (and some times from a list of their related agents) and after each suggestion, there is negotiation.
If there would not be network effect in the data, I would use simple heuristics, which would measure the diff of each feature. However, in network effect world this does not work very well, because it will make many agents unable to co-operate.
I was thinking of something, where some features are preferred features by the agents in different circumstances (depending on the other agents), this will enable some elasticity to the system and IMO avoid the simple problems, of which many recommendation systems suffer (lack of context awareness; humans do not like the likes of other people, we have flexible preferences, which depend upon our mood).
Since this sounds like a very generic problem, it seems like I have just discovered a problem, which has been elegantly solved by someone else from the academia years ago, but I just don't know the name for this concept.
Instead of depending on strict evaluation criteria (only accept diff X per feature), how to choose evaluation function to make the agents more co-operative? I was thinking something, where diff has to be narrow for at least one feature, but may be big for three worst matching features; so that there would be essence of few features and diversity of several.
EDIT: As I have spent some more time with this problem, I have a hunch about which issues might be related:
- In real world networks, which depend upon energy efficient co-operation and signaling, scale-free networks seem to emerge; effective Agent-Based Model in network effect scenario should probably form co-operation networks, which follows some kind of Fitness Function.
- Due to scale-free networks, it should be assumed, that the Agents using essential features (which are common in data) should be ranked higher and Agents using diverse features (which are rare in data) should be ranked lower; in scale-free networks (like Facebook) it has been observed, that all but 3% of nodes form edges to more popular nodes: this makes signaling efficient.
- As a reference, in deep learning Capsule Networks have had some state-of-the-art success with this similar hierarchical "recommendation for upper tiers" mentality (there are much more lower layer features, than upper layer features).
So, I have this idea, that when we use Agent-Based Models for solving problems with network effects, the evaluation function should evaluate Agents skill of passing the problem for more popular Agent, which could delegate it to the best Agent or attempt to solve it on it's own.