1
$\begingroup$

I've recently come across the client-server model. From my understanding, the client requests the server, to which the server responds with a response. In this case, both the request and responses are vectors.

In reinforcement learning, the agent communicates with the environment via an "action", to which the environment sends a scalar reward signal. The "goal" is to maximize this scalar reward signal in long run.

Is there a more formal way to frame this problem such the agent searches the internet for obtaining more information about certain topic?

$\endgroup$
1
$\begingroup$

To paraphrase: "Is there an analogy between client/server in web development and agent/environment in reinforcement learning?"

The answer is "not really". There is no useful analogy here that allows any insight into RL from web server knowledge or vice versa.

However, you could set up an agent where the goal was to collect information, and the available actions were to make web requests. Clearly in order to to this, you would need to make use of the client/server model for web servers, with the agent having control over the client web requests, and the environment being the network and servers of the world wide web.

There are some very hard challenges to construct an open-ended "web assistant" agent. Here are a couple that I can think of:

  • How to describe actions? Starting with raw web requests composed as strings would likely be very frustrating. Probably you would simplify and have a first action be a call to a search engine with some variation of the topic description, and then decisions about which links to follow, or perhaps whether to refine the search to better fetch sites related to the topic as it is being built.

  • How to create a model of reward for collecting information? The first major stumbling block would be to measure the amount of useful information that the agent had found on any request.

I think with current levels of Natural Language Processing, setting an agent free to discover information according to some goal from a text topic description is a very hard task, beyond cutting edge research. It would definitely be unreasonable to expect any such agent to end up with any resemblance to "understanding" subject matter from text. The agent would have very little ability to tell the difference between accurate facts or lies, or just grammatically correct gibberish.

One interesting idea for agents trying to learn unsupervised from exploring an environment is creating a reward signal from data compression. An agent will have learned something useful about its environment, if on processing new information, it is able to compress its existing world model more efficiently. This is basic concept behind ideas of general learning agents from Jürgen Schmidhuber, Marcus Hutter and others. Research into this idea could be a driver towards creating more generic AI learning systems - however, it is one idea amongst many in AI, and so far is research-only, it has not yet led to anything as practical as an AI web-searching assistant.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.