Even if you could do that (which I believe is a long way off), what would be the point?
If I understand you correctly, you want an AI system to learn through observation of human beings what their 'rules of interaction' are. It sees a person killing someone else, and then that person is punished by the community, so the AI learns that killing people is not right. However, that is already codified in laws... So what it would pick up are social behaviours, which hopefully be things like "be nice to other people", "don't do anybody any harm", "be honest and truthful", etc.
The first question then is how an AI could assess events[*] as to whether they are 'good' or 'bad'. If everybody lies and steals, that would be learnt as normal behaviour, and the AI would not be able to pick up that most people would see this as 'bad'[**]. Causal relations are also hard to grasp. Somebody steals something. Then later someone else buys him a drink. So stealing stuff means other people will buy you drinks? These are really hard problems to solve. You need to know about people's motivations, and the multitude of 'threads' of interaction happening at the same time, even in a very limited area.
So, recognising events and causal links, plus a moral evaluation of them is pretty difficult. I don't think we will get there anytime soon. Unsupervised learning of behavioural is also pretty difficult, as you only have unlabelled observations and no real criteria to classify them. Plus, many actions are morally ambiguous. Killing people is generally seen as bad. What about the officers who tried to assassinate Hitler in 1944? Life is complex, and our artificial models are not anywhere near that.
So even if you were able to do all that, what would you end up with? An AI system that has picked up a lot of unwritten rules about human behaviour, and then postulates that as laws? So everyone has to behave the same way? I just don't see the point of that, even as a thought experiment.
[*] leaving aside here the question how you determine what an 'event' is in the first place
[**] Please note that even if people steal things, they steal can view theft as a bad thing.