-
Notifications
You must be signed in to change notification settings - Fork 13
Open
Description
What is the purpose of this processing code in prepro_fn_gat_bert()?
order_match = {}
count = 1
i = 0
for sents in [' '.join(source_sent)]:
sent_words = sents.split(' ')
if len(sent_words) > 0:
order_match[i] = list(range(count, count + align[sent_words[0]]))
count += align[sent_words[0]]
i += 1
for word in sent_words[1:]:
new_word = ' ' + word # if use bpe
#new_word = word
order_match[i] = list(range(count, count + align[new_word]))
# test_order_match[new_word] = [count, count + align[new_word]]
count += align[new_word]
i += 1
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels