0
$\begingroup$

tldr; I am researching/looking for vec2text methods that work (a) without training a special model to guide the mutations, and (b) without access to the model weights.

I am doing some experiments in navigating the encoded space of an encoder-only text embedding model (such as OpenAI's text-embedding-ada-002, but not specific to this model), with the limitation that I do not have access to the weights. I am trying to find some heuristics to "travel" towards a target embedding within the space.

Note: this is about text embedding (e.g sentences, maybe paragraphs), not word embedding.

Some ideas I currently have:

  • Running an evolutionary algorithm to mutate the text in (text, embedding) pairs as individuals, and using dist(embedding, target embedding) as an objective.

  • At each step, locally finding "directional" vectors by taking the delta of two steps combined with some adjective, and associating meaning to the direction.

  • Applying a direction repeatedly if it gets me closer to the target.

  • Asking an LLM to {mutate, combine} an/two individuals' text, and also to tell me the semantic difference (giving meaning to the direction).

  • Inspiration/existing research: I am aware of the Vec2Text from Text Embeddings Reveal (Almost) As Much As Text, but i am specifically looking to do as well as possible without training a special model to guide the mutations.

  • Q: Is there any existing research on this sort of thing?

  • Q: Any better mutation heuristics? Other approaches entirely?

  • Q: I vaguely aware of the Manifold Hypothesis, are there ways to detect if an embedding is far from the manifold? Or a way to "get back" to the manifold?

$\endgroup$

1 Answer 1

1
$\begingroup$

The embedding space of different models may be more similar than you think. Your problem reminds me a bit of blackbox adversarial attacks, where you want to e.g., find an adversarial example targeting some API to induce an output without having access to model weights.

For example, Zou et al., 2023 and Wallace et al., 2019 both suggest finding adversarial examples with whitebox access to an ensemble of models (through gradient steps + discrete updates), with the goal being to induce certain types of outputs (e.g., toxic/harmful, different sentiment etc.). They then notice that these examples transfer to models that you only have API access to. This is pretty directly transferrable to your setting: take gradient steps + discrete updates (as described in the papers) to match the embeddings of an ensemble of whitebox models, then this might end up transferring to whatever blackbox API you're targeting.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .