degeneration

mentioned <b>fine-grained</b> control

train on seq of raw text prepended with <b>control codes</b>

famous for story generation

new loss

seq-level &amp; token-level

new loss

new loss

random sample

beam search

heuristic random sample

heuristic beam search - beast first beam search

heuristic guided decoding strategy

raise diversity

<b>fine-grained </b>control including topic, end, repeatness

random sample+beam search

select based on conditional entropy

conbine <b>similarity</b> and beam search

<b>intrinsic uncertainty </b>could be solved by increase num_beam<br>harder to be trapped in <b>local optimum</b>

<b>different task&nbsp;</b>would have different optimal decoding strategy

<b>high quality and high prob positive/negative correlation</b> upon different task

appropriate<b> information entropy </b>is decisive upon quality (encourage model to generate tokens based on <b>information</b>)<br>information theory related

idea : gamma sampling treat equally every&nbsp;homoionyms<br>this new sample method treat homoionyms differently sorted by their <b>cosine similarity towards keywords appeared in source text</b>

<span style="font-size: inherit;">change topic_tokens to key_words directly : works</span><br>

add key_gamma : doesn't work

change probs before gamma sampling

delete the extreme condition dealing : works