What is the difference between a normal processor and a processor designed for AI?
Parallel processing is very important when computing ANNs. Though they are not designed for AIs, GPUs are widely used in machine learning because of their higher capability of paralel processing and it is due to their higher number of cores comparing CPUs.
All in all, the dedicated hardware should have GPU-like architecture.
Specialized AI hardware takes advantage of highly parallelizable nature of many neural network designs. GPUs are designed for pixel crunching - coincidentally very paralelisable too. This is why they often offer orders of magnitude better performance (in NN tasks) than CPUs. That being said, GPUs have shortcomings to. First, many of GPU's built in features are not utilized in neural network applications- in many cases accurate floating point operations are not necessary, data for simple calculations involved in NNs. If you remove all the unused features from GPUs and use that extra space for more compute units, you get something like Googles' https://cloud.google.com/blog/big-data/2017/05/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu TPU.