degeneration

mentioned fine-grained control

train on seq of raw text prepended with control codes

famous for story generation

CTRL-baseline

new loss

seq-level & token-level

unlikelihood training-baseline

simCTG

CTLoss

learning-based

topp

topk

temperature

random sample

create forbid set

beam search

basic-baseline

heuristic random sample

heuristic beam search - beast first beam search

heuristic guided decoding strategy

heuristic

raise diversity

gamma sampling

random sample+beam search

composition decoding

select based on conditional entropy

typical decoding

conbine similarity and beam search

direct beam search

decoding-baseda.k.a. guided decoding

prompt?

other guided decoding

previous question answering

modeling decoding strategy selection

future

intrinsic uncertainty could be solved by increase num_beamharder to be trapped in local optimum

\"n best search\"

different task would have different optimal decoding strategy

high quality and high prob positive/negative correlation upon different task

decoding limitation

appropriate information entropy is decisive upon quality (encourage model to generate tokens based on information)information theory related

high quality!=high prob

discussion

idea : gamma sampling treat equally every homoionymsthis new sample method treat homoionyms differently sorted by their cosine similarity towards keywords appeared in source text

change topic_tokens to key_words directly : works

add key_gamma : doesn't work

change probs before gamma sampling

delete the extreme condition dealing : works

gamma-enhanced sample

answer towards previous question

new idea

degeneration