I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity.
!python create_pretraining_data.py \
--input_file=/input_path/input_file.txt \
--output_file=/tf_path/tf_examples.tfrecord \
--vocab_file=/vocab_path/uncased_L-12_H-768_A-12/vocab.txt \
--do_lower_case=True \
--max_seq_length=128 \
--max_predictions_per_seq=20 \
--masked_lm_prob=0.15 \
--random_seed=12345 \
--dupe_factor=5
!python run_pretraining.py \
--input_file=/tf_path/tf_examples.tfrecord \
--output_dir=pretraining_output \
--do_train=True \
--do_eval=True \
--bert_config_file=/bert_path/uncased_L-12_H-768_A-12/bert_config.json \
--init_checkpoint=/bert_path/uncased_L-12_H-768_A-12/bert_model.ckpt\
--train_batch_size=32 \
--max_seq_length=128 \
--max_predictions_per_seq=20 \
--num_train_steps=20 \
--num_warmup_steps=10 \
--learning_rate=2e-5
I have run a pre-trained BERT model with some domain of corpora from scratch. I have got the checkpoints and graph.pbtxt file from the code above. But I am unaware on how to use those files for evaluating semantic text similarity test file.