Often times I see the term deep reinforcement learning to refer to RL algorithms that use neural networks, regardless of whether or not the networks are deep.
For example, PPO is often considered a deep RL algorithm, but using a deep network is not really part of the algorithm. In fact, the example they report in the paper says that they used a network with only 2 layers.
This SIGGRAPH project (DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills) has the name deep in it and the title even says 'deep reinforcement learning', but if you read the paper, you'll see that their network uses only 2 layers.
Again, the paper Learning to Walk via Deep Reinforcement Learning by researchers from Google and Berkeley, contains deep RL in the title, but if you read the paper, you'll see they used 2 hidden layers.
Another SIGGRAPH project with deep RL in the title. And, if you read it, surprise, 2 hidden layers.
In the paper Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, if you read table 1 with the hyperparameters, they also used 2 hidden layers.
Is it standard to just call deep RL to any RL algorithm that uses a neural net?