Are the capabilities of connectionist AI and symbolic AI the same?
No, not usually. Why not usually? Neural networks (connectionist AI) are usually used for inductive reasoning (i.e. the process of generalizing given a finite set of observations), while symbolic AI is usually used for deduction (i.e. to logically derive conclusions from premises).
What is inductive reasoning? Let's say that all the birds that you have observed so far in your life fly, so your inductive thought is that all birds must fly, although you haven't seen all birds, so there could be exceptions (like penguins).
What is deductive reasoning? Let's say that you know that all humans are mortal (there's no exception). You know that Socrates is a human. So, you logically deduce that Socrates is mortal. If that was not the case, then either the premise was wrong or maybe Socrates is not a human.
We can express any MLP in terms of algebraic expressions. And the expressions can be considered as symbolic-AI.
So, can I infer that symbolic AI algorithms can theoretically approximate any bounded continuous function?
Now, if MLPs were a subset of symbolic AI, then we could conclude that symbolic AI can approximate bounded continuous functions. However, the definition of symbolic AI is usually restricted to knowledge-based and logic-based systems, so systems that write (e.g. using propositional logic) premises or facts (in knowledge bases) to deduce conclusions (other facts) from them. So, although $f(x) = \sigma(ax + b)$ (which can represent e.g. a perceptron) is a function with symbols $a$, $x$, $b$ and $\sigma(\cdot)$, these symbols are not used in the same way as the symbols in symbolic AI. They are variables that do not represent premises or conclusions.
If not, then why can't there be a one-one mapping between MLP and symbolic-AI algorithm?
I don't know if there can be a one-to-one mapping between some symbolic AI and neural networks.
However, it is possible to combine the two approaches. For example, in the context of knowledge graphs, which can be viewed as a way to represent facts and relations between those facts as a graph, we can learn embeddings (using machine learning), which can later be used to perform inductive reasoning on the knowledge graph.
There are other examples of attempts to combine symbolic AI with machine learning and neural networks. A famous example is the Markov Logic Network (MLN) (by Richardson and Domingos), which combines first-order logic with probabilistic graphical models. A related approach is used, for example, used in OpenCog, which is a software platform for AI and AGI. In fact, I think it is widely believed that inductive and deductive reasoning are both necessary for an AGI. The AGI needs inductive reasoning for situations that involve uncertainty (most cases) and deductive reasoning for the remaining cases (e.g. doing math, i.e. we expect an AGI to be able to prove theorems, as we do). Another example is a combination of knowledge graphs with MLNs.
