生成式AI文本密码：Transformer参数全解码​ 原创

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
#创建模型和分词器
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
#将输入提示分词为ID序列
prompt = "Artificial intelligence is"
inputs = tokenizer(prompt, return_tensors="pt")
# 将输出作为一系列标记ID生成
output = model.generate(
 **inputs,
 max_length=50,
 num_return_sequences=1,
 temperature=1.0,
 top_k=50,
 top_p=1.0,
 repetition_penalty=1.0,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
)
#将标记ID转换为文本字符串
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(f"Prompt: {prompt}")
print("Generated Text:")
print(generated_text)

Prompt: Artificial intelligence is
Generated Text:
Artificial intelligence is used in the production of technology, the delivery of
which is determined by technological change. For example, an autonomous car can
change its steering wheel to help avoid driving traffic. In the case of artificial
intelligence, this can change what consumers

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
prompt = "The future of artificial intelligence is"
inputs = tokenizer(prompt, return_tensors="pt")
# 生成不同温度值的文本
temperatures = [0.2, 0.5, 1.0, 1.5]
print(f"Prompt: {prompt}")
for temp in temperatures:
 print()
 print(f"Temperature: {temp}")
 output = model.generate(
 **inputs,
 max_length=100,
 num_return_sequences=1,
 temperature=temp,
 top_k=50,
 top_p=1.0,
 repetition_penalty=1.0,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
 )
 generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
 print("Generated Text:")
 print(generated_text)

Prompt: The future of artificial intelligence is
Temperature: 0.2
Generated Text:
The future of artificial intelligence is uncertain. The future of artificial
intelligence is uncertain.
The future of artificial intelligence is uncertain. The future of artificial
intelligence is uncertain.
The future of artificial intelligence is uncertain. The future of artificial
intelligence is uncertain.
The future of artificial intelligence is uncertain. The future of artificial
intelligence is uncertain.
The future of artificial intelligence is uncertain. The future of artificial
intelligence is uncertain.
The future of artificial intelligence is uncertain. The future
Temperature: 0.5
Generated Text:
The future of artificial intelligence is uncertain.
"There is a lot of work to be done on this," said Eric Schmitt, a professor
of computer science and engineering at the University of California, Berkeley.
"We're looking for a way to make AI more like computers. We need to take a step
back and look at how we think about it and how we interact with it."
Schmitt said he's confident that artificial intelligence will eventually be
able to do more than
Temperature: 1.0
Generated Text:
The future of artificial intelligence is not yet clear, however."
"Is the process that we are trying to do through computer vision and the ability to
look at a person at multiple points without any loss of intelligence due to not
seeing a person at multiple points?" asked Richard. "I also think the people who
are doing this research are extremely interesting to me due to being able to see
humans at a range of different points in time. In particular, they've shown how
to do a pretty complex
Temperature: 1.5
Generated Text:
The future of artificial intelligence is an era to remember as much as Google in
search results, particularly ones not supported by much else for some years -- and
it might look like the search giant is now just as good without artificial
intelligence. [Graphic image from Shutterstock]

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
prompt = "The best way to learn programming is"
inputs = tokenizer(prompt, return_tensors="pt")
#使用不同top_k值生成文本
top_k_values = [5, 20, 50]
print(f"Prompt: {prompt}")
for top_k in top_k_values:
 print()
 print(f"Top-K = {top_k}")
 output = model.generate(
 **inputs,
 max_length=100,
 num_return_sequences=1,
 temperature=1.0,
 top_k=top_k,
 top_p=1.0,
 repetition_penalty=1.0,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
 )
 generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
 print("Generated Text:")
 print(generated_text)
# 使用不同top_p值生成文本
top_p_values = [0.5, 0.7, 0.9]
for top_p in top_p_values:
 print()
 print(f"Top-P = {top_p}")
 output = model.generate(
 **inputs,
 max_length=100,
 num_return_sequences=1,
 temperature=1.0,
 top_k=0,
 top_p=top_p,
 repetition_penalty=1.0,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
 )
 generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
 print("Generated Text:")
 print(generated_text)
如果运行此代码，你可能会看到如下输出内容：
Prompt: The best way to learn programming is
Top-K = 5
Generated Text:
The best way to learn programming is to be able to learn the basics in a very short
amount of time, and then learn to use them effectively and quickly.
If you want to be a successful programmer in this way, you should learn to use the
techniques in the above video to learn the basics of programming.
If you want to learn to code more effectively, you can also get more experienced
programmers by doing the following:
Learning to Code
Learning to code is very
Top-K = 20
Generated Text:
The best way to learn programming is to learn it.
In order to get started with Ruby you're going to have to make a few mistakes, some
of them can be fairly obvious.
First of all, you're going to have to write a function that takes in a value. What
this means is that you're going to make a new instance of the Ruby function. You can
read more about this in Part 1 of this course, or just try it out from the REPL.
Top-K = 50
Generated Text:
The best way to learn programming is to become familiar with the language and the
software. One of the first and most common forms of programming is to create,
modify, and distribute code.
However, there are very few programming libraries that can provide us with all
that we need.
The following sample programming program uses some of the above, but does not show
the best way to learn programming. It was written in Java and in C or C++.
The original source code is
Top-P = 0.5
Generated Text:
The best way to learn programming is to be able to create a tool for you. That's
what I do.
That's why I'm here today.
I'm here to talk about the basics of programming, and I'm going to tell you how to
learn programming.
I'm here to talk about learning programming.
It's easy to forget that you don't have to know how to program. It's easy to forget
that you don't have to know how
Top-P = 0.7
Generated Text:
The best way to learn programming is to practice programming. Learn the principles
of programming by observing and performing exercises.
I used to work in a world of knowledge which included all sorts of things, and was
able to catch up on them and understand them from their perspective. For instance, I
learned to sit up straight and do five squats. Then, I would have to practice some
type of overhead training. I would try to learn the best technique and add that to
my repertoire.
What
Top-P = 0.9
Generated Text:
The best way to learn programming is to become a good hacker. Don't use any
programming tools. Just a regular dot-com user, an occasional coding learner, and
stick with it.
— Victoria E. Nichols

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
prompt = "Once upon a time, there was a"
inputs = tokenizer(prompt, return_tensors="pt")
# 使用不同的重复惩罚生成文本
penalties = [1.0, 1.2, 1.5, 2.0]
print(f"Prompt: {prompt}")
for penalty in penalties:
 print()
 print(f"Repetition penalty: {penalty}")
 output = model.generate(
 **inputs,
 max_length=100,
 num_return_sequences=1,
 temperature=0.3,
 top_k=50,
 top_p=1.0,
 repetition_penalty=penalty,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
 )
 generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
 print("Generated Text:")
 print(generated_text)

Prompt: Once upon a time, there was a
Repetition penalty: 1.0
Generated Text:
Once upon a time, there was a great deal of confusion about what was going on. The
first thing that came to mind was the fact that the government had already been in
place for a long time, and that the government had been in place for a long time.
And it was clear that the government had been in place for a long time. And it was
clear that the government had been in place for a long time. And it was clear that
the government had been in place for a long
Repetition penalty: 1.2
Generated Text:
Once upon a time, there was a great deal of talk about the possibility that this
would be an opportunity for us to see more and better things in our lives. We had
been talking on Facebook all day long with people who were interested in what we
could do next or how they might help others find their own way out."
"We've always wanted to make sure everyone has access," he continued; "but it's not
like you can just go into your room at night looking around without seeing
Repetition penalty: 1.5
Generated Text:
Once upon a time, there was a man who had been called to the service of God. He
came and said: "I am an apostle from Jerusalem." And he answered him with great joy,
saying that it is not possible for me now in this life without having received
Jesus Christ as our Lord; but I will be saved through Him alone because my Father
has sent Me into all things by His Holy Spirit (John 1).
The Christian Church teaches us how much more than any other religion can
Repetition penalty: 2.0
Generated Text:
Once upon a time, there was a man who had been sent to the city of Nausicaa by his
father. The king's son and brother were killed in battle at that place; but when
he returned with them they found him dead on their way back from war-time.[1]
The King gave orders for an expedition against this strange creature called "the
Gorgon," which came out into space during one night after it attacked Earth[2]. It
is said that these creatures

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
prompt = "The secret to happiness is"
inputs = tokenizer(prompt, return_tensors="pt")
# 使用贪婪解码与采样生成文本
print(f"Prompt: {prompt}\n")
print("Greedy Decoding (do_sample=False):")
output = model.generate(
 **inputs,
 max_length=100,
 num_return_sequences=1,
 temperature=1.0,
 top_k=50,
 top_p=1.0,
 repetition_penalty=1.0,
 do_sample=False,
 pad_token_id=tokenizer.eos_token_id,
)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print("Generated Text:")
print(generated_text)
print()
print("Sampling (do_sample=True):")
output = model.generate(
 **inputs,
 max_length=100,
 num_return_sequences=1,
 temperature=1.0,
 top_k=50,
 top_p=1.0,
 repetition_penalty=1.0,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print("Generated Text:")
print(generated_text)

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")

prompt = "The key to successful machine learning is"
inputs = tokenizer(prompt, return_tensors="pt")

#使用贪婪解码与采样生成文本
print(f"Prompt: {prompt}\n")
outputs = model.generate(
 **inputs,
 num_beams=5, # 要使用的光束数量
 early_stopping=True, # 当所有光束都完成时停止
 no_repeat_ngram_size=2, # 避免重复n-gram
 num_return_sequences=3, # 返回多个序列
 max_length=100,
 temperature=1.5,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
)
for idx, output in enumerate(outputs):
 generated_text = tokenizer.decode(output, skip_special_tokens=True)
 print(f"Generated Text ({idx+1}):")
 print(generated_text)

Prompt: The key to successful machine learning is

Generated Text (1):
The key to successful machine learning is to be able to learn from the world around
you. It is our job to make sure that we are learning from people, rather than just
from machines.

So, let's take a step back and look at how we can learn. Here's a list of the tools
we use to help us do that. We're going to go over a few of them here and give you
a general idea of what they are and how you can use them to create

Generated Text (2):
The key to successful machine learning is to be able to learn from the world around
you. It is our job to make sure that we are learning from people, rather than just
from machines.

So, let's take a step back and look at how we can learn. Here's a list of the tools
we use to help us do that. We're going to go over a few of them here and give you
a general idea of what they are and how you can use them and what

Generated Text (3):
The key to successful machine learning is to be able to learn from the world around
you. It is our job to make sure that we are learning from people, rather than just
from machines.

So, let's take a step back and look at how we can learn. Here's a list of the tools
we use to help us do that. We're going to go over a few of them here and give you
a general idea of what they are and how they work. You can use