Understanding how an algorithm works can be realized with two possibilities. At first, with Opensource software. That is a well known method and prevents proprietary non-documented code. Opensource software alone doesn't solve the problem, as seen by @k.c. sayz 'k.c sayz' correctly. For example, if we are implementing a neural network as opensource, it remains a blackbox which is not predictable, especially for non-experts.
The shared language between legal laws and engineering capabilities is the natural language, especially English. So the engineers have to create their system in a way, that it communicates with the world in normal English. That means not, that the car speaks like K.I.T.T., it means only that the interface for controlling the car has commands like “start”, “stop” and “drive slower”. If the car detects with his neural network a pedestrian, than it should print out on the console a simple “pedestrian detected”. That seems natural, but most programs today printing out “memoryadress $3400 = -65” or “sigmoid activation function is set to sin(4-x)”.
In a potential law case it makes sense to cite status-messages of the onboard computer. Because if they are written in normal English, lawyers will understand it. How the natural language output was produced is in the scope of the engineers and is only a detail question.