A quite effective way to solve Wordle is to maintain an active list of all five-letter words that meet criteria so far, choose one of them completely at random, then update the list of possible words based on new knowledge from the response.
A slightly smarter way would be to pick words from the subset of remaining ones that should maximise information gathered from the response on average. I believe I read a headline recently that "crane" is a good starting English word for that reason.
The trouble for you here is that those approaches do not involve ML in any form. Relatively simple logic and analysis are already pretty good at playing Wordle automatically.
However, to stick with the question as asked, your problem is how to get a machine learning algorithm to choose between maybe 10,000 options.
You have a couple of choices:
Brute force. Number the options, have the algorithm choose directly. If you do this, I recommend that you also filter out logically impossible words, as that makes the problem far easier. The 10,000 options may seem large at first, but actually this should be within reach of a personal/hobby setup. It will work mainly because it is possible to simulate the results of games very fast. So generating the training data within RL will be fast, the agent can get to see millions of games in a reasonable time frame, training with each target word thousands of times on average.
Actions represent additional constraints on word choice. For a simple example, you could have the agent decide whether the first unknown letter it wants to try is a vowel or consonant = just two actions. You can use that to further restrict word choice from the remaining words. This influences an otherwise random word selector, and the agent should learn situations in which to pick words with certain traits.
Actions describe a "word vector", and the selector picks the closest valid word to that vector, breaking ties randomly. In order to do this, you need to generate some short vector of each word in the dictionary. For a simple example* you might choose [part_of_speech, popularity] to qualify each word, scoring part_of_speech as 1 for noun, -1 for verb and 0 for everything else and popularity as 1 for common word, -1 for rare word, and 0 for moderately frequent. Then your action output would be a choice from the 9 combinations possible in this vector.
None of these approaches are simple, so not great if you are just starting in AI/ML, and I would recommend you tackle a simpler toy problem or two using RL before trying to write your Wordle training routine. Going straight to Wordle without solving smaller problems first is likely to cause a lot of incorrect decisions on your part that you won't spot and will frustrate you.
* I have given a very simple example, but this could logically lead to learned word embeddings, which are an important topic in natural language processing. These are typically quite large real-valued vectors though (maybe 64 or 128 elements) which would be an even larger space to learn about than the fixed list of words. So full word embeddings like word2vec and Glove are not really suitable for the Wordle puzzle.
