Skip to content

Commit 438cfb2

Browse files
am831awharrison-28
authored and
SOE-YoungS
committed
Python: Add Google PaLM connector with chat completion and text embedding (microsoft#2258)
### Motivation and Context Implementation of Google PaLM connector with chat completion, text embedding and three example files to demonstrate their functionality. Closes microsoft#2098 I also opened a PR for text completion microsoft#2076 <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description **What's new:** 1. Implemented Google Palm connector with chat completion and text embedding 2. Added 3 example files to `python/samples/kernel-syntax-examples` 3. The example files show PaLM chat in action with skills, memory, and just with user messages. The memory example uses PaLM embedding as well 5. Added integration tests to test chat with skills and embedding with kernel.memory functions 6. Added unit tests to verify successful class initialization and correct behavior from class functions **There are some important differences between google palm chat and open ai chat:** 1. PaLM has two functions for chatting, `chat` and `reply`. The chat function in google's genai library starts a new conversation, and reply continues the conversation. Reply is an attribute of the response object returned by chat. So an instance of the `GooglePalmChatCompletion` class needs a way to determine which function to use, which is why I introduced a private attribute to store the response object. See https://developers.generativeai.google/api/python/google/generativeai/types/ChatResponse 2. PaLM does not use system messages. Instead, the chat function takes a parameter called context. It serves the same purpose, to prime the assistant with certain behaviors and information. So when the user passes a system message to `complete_chat_async`, it is passed to `chat` as the context parameter. 3. Semantic memory works with the chat service as long as the user creates a chat prompt template. The prompt containing the memory needs to be added to the chat prompt template as a system message. See `python\samples\kernel-syntax-examples\google_palm_chat_with_memory.py` for more details. If the only purpose of `complete_async` in `GooglePalmChatCompletion` is to send memories + user messages to the chat service as a text prompt, then `complete_async` is not fulfilling its intended purpose. A possible solution would be to send the text prompt as a request to the text service within `complete_async`. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 Currently no warnings, there was 1 warning when first installing genai with `poetry add google.generativeai==v0.1.0rc2` from within poetry shell: "The locked version 0.1.0rc2 for google-generativeai is a yanked version. Reason for being yanked: Release is marked as supporting Py3.8, but in practice it requires 3.9". We would need to require later versions of python to fix it. --------- Co-authored-by: Abby Harrison <[email protected]> Co-authored-by: Abby Harrison <[email protected]> Co-authored-by: Abby Harrison <[email protected]>
1 parent 0b886d4 commit 438cfb2

File tree

13 files changed

+1077
-50
lines changed

13 files changed

+1077
-50
lines changed

‎python/poetry.lock

+244-40
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎python/samples/kernel-syntax-examples/google_palm_chat.py

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
5+
import semantic_kernel as sk
6+
import semantic_kernel.connectors.ai.google_palm as sk_gp
7+
from semantic_kernel.connectors.ai.chat_request_settings import ChatRequestSettings
8+
9+
10+
async def chat_request_example(api_key):
11+
palm_chat_completion = sk_gp.GooglePalmChatCompletion(
12+
"models/chat-bison-001", api_key
13+
)
14+
settings = ChatRequestSettings()
15+
settings.temperature = 1
16+
17+
chat_messages = list()
18+
user_mssg = "I'm planning a vacation. Which are some must-visit places in Europe?"
19+
chat_messages.append(("user", user_mssg))
20+
answer = await palm_chat_completion.complete_chat_async(chat_messages, settings)
21+
chat_messages.append(("assistant", str(answer)))
22+
user_mssg = "Where should I go in France?"
23+
chat_messages.append(("user", user_mssg))
24+
answer = await palm_chat_completion.complete_chat_async(chat_messages, settings)
25+
chat_messages.append(("assistant", str(answer)))
26+
27+
context_vars = sk.ContextVariables()
28+
context_vars["chat_history"] = ""
29+
context_vars["chat_bot_ans"] = ""
30+
for role, mssg in chat_messages:
31+
if role == "user":
32+
context_vars["chat_history"] += f"User:> {mssg}\n"
33+
elif role == "assistant":
34+
context_vars["chat_history"] += f"ChatBot:> {mssg}\n"
35+
context_vars["chat_bot_ans"] += f"{mssg}\n"
36+
37+
return context_vars
38+
39+
40+
async def main() -> None:
41+
api_key = sk.google_palm_settings_from_dot_env()
42+
chat = await chat_request_example(api_key)
43+
print(chat["chat_history"])
44+
return
45+
46+
47+
if __name__ == "__main__":
48+
asyncio.run(main())

‎python/samples/kernel-syntax-examples/google_palm_chat_with_memory.py

+142
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
from typing import Tuple
5+
6+
import semantic_kernel as sk
7+
import semantic_kernel.connectors.ai.google_palm as sk_gp
8+
9+
kernel = sk.Kernel()
10+
apikey = sk.google_palm_settings_from_dot_env()
11+
palm_text_embed = sk_gp.GooglePalmTextEmbedding("models/embedding-gecko-001", apikey)
12+
kernel.add_text_embedding_generation_service("gecko", palm_text_embed)
13+
palm_chat_completion = sk_gp.GooglePalmChatCompletion("models/chat-bison-001", apikey)
14+
kernel.add_chat_service("models/chat-bison-001", palm_chat_completion)
15+
kernel.register_memory_store(memory_store=sk.memory.VolatileMemoryStore())
16+
kernel.import_skill(sk.core_skills.TextMemorySkill())
17+
18+
19+
async def populate_memory(kernel: sk.Kernel) -> None:
20+
# Add some documents to the semantic memory
21+
await kernel.memory.save_information_async(
22+
"aboutMe", id="info1", text="My name is Andrea"
23+
)
24+
await kernel.memory.save_information_async(
25+
"aboutMe", id="info2", text="I currently work as a tour guide"
26+
)
27+
await kernel.memory.save_information_async(
28+
"aboutMe", id="info3", text="My favorite hobby is hiking"
29+
)
30+
await kernel.memory.save_information_async(
31+
"aboutMe", id="info4", text="I visitied Iceland last year."
32+
)
33+
await kernel.memory.save_information_async(
34+
"aboutMe", id="info5", text="My family is from New York"
35+
)
36+
37+
38+
async def search_memory_examples(kernel: sk.Kernel) -> None:
39+
questions = [
40+
"what's my name",
41+
"what is my favorite hobby?",
42+
"where's my family from?",
43+
"where did I travel last year?",
44+
"what do I do for work",
45+
]
46+
47+
for question in questions:
48+
print(f"Question: {question}")
49+
result = await kernel.memory.search_async("aboutMe", question)
50+
print(f"Answer: {result}\n")
51+
52+
53+
async def setup_chat_with_memory(
54+
kernel: sk.Kernel,
55+
) -> Tuple[sk.SKFunctionBase, sk.SKContext]:
56+
"""
57+
When using Google PaLM to chat with memories, a chat prompt template is
58+
essential; otherwise, the kernel will send text prompts to the Google PaLM
59+
chat service. Unfortunately, when a text prompt includes memory, chat
60+
history, and the user's current message, PaLM often struggles to comprehend
61+
the user's message. To address this issue, the prompt containing memory is
62+
incorporated into the chat prompt template as a system message.
63+
Note that this is only an issue for the chat service; the text service
64+
does not require a chat prompt template.
65+
"""
66+
sk_prompt = """
67+
ChatBot can have a conversation with you about any topic.
68+
It can give explicit instructions or say 'I don't know' if
69+
it does not have an answer.
70+
71+
Information about me, from previous conversations:
72+
- {{$fact1}} {{recall $fact1}}
73+
- {{$fact2}} {{recall $fact2}}
74+
- {{$fact3}} {{recall $fact3}}
75+
- {{$fact4}} {{recall $fact4}}
76+
- {{$fact5}} {{recall $fact5}}
77+
78+
""".strip()
79+
80+
prompt_config = sk.PromptTemplateConfig.from_completion_parameters(
81+
max_tokens=2000, temperature=0.7, top_p=0.8
82+
)
83+
prompt_template = sk.ChatPromptTemplate( # Create the chat prompt template
84+
"{{$user_input}}", kernel.prompt_template_engine, prompt_config
85+
)
86+
prompt_template.add_system_message(sk_prompt) # Add the memory as a system message
87+
function_config = sk.SemanticFunctionConfig(prompt_config, prompt_template)
88+
chat_func = kernel.register_semantic_function(
89+
None, "ChatWithMemory", function_config
90+
)
91+
92+
context = kernel.create_new_context()
93+
context["fact1"] = "what is my name?"
94+
context["fact2"] = "what is my favorite hobby?"
95+
context["fact3"] = "where's my family from?"
96+
context["fact4"] = "where did I travel last year?"
97+
context["fact5"] = "what do I do for work?"
98+
99+
context[sk.core_skills.TextMemorySkill.COLLECTION_PARAM] = "aboutMe"
100+
context[sk.core_skills.TextMemorySkill.RELEVANCE_PARAM] = 0.6
101+
102+
context["chat_history"] = ""
103+
104+
return chat_func, context
105+
106+
107+
async def chat(
108+
kernel: sk.Kernel, chat_func: sk.SKFunctionBase, context: sk.SKContext
109+
) -> bool:
110+
try:
111+
user_input = input("User:> ")
112+
context["user_input"] = user_input
113+
except KeyboardInterrupt:
114+
print("\n\nExiting chat...")
115+
return False
116+
except EOFError:
117+
print("\n\nExiting chat...")
118+
return False
119+
120+
if user_input == "exit":
121+
print("\n\nExiting chat...")
122+
return False
123+
124+
answer = await kernel.run_async(chat_func, input_vars=context.variables)
125+
context["chat_history"] += f"\nUser:> {user_input}\nChatBot:> {answer}\n"
126+
127+
print(f"ChatBot:> {answer}")
128+
return True
129+
130+
131+
async def main() -> None:
132+
await populate_memory(kernel)
133+
await search_memory_examples(kernel)
134+
chat_func, context = await setup_chat_with_memory(kernel)
135+
print("Begin chatting (type 'exit' to exit):\n")
136+
chatting = True
137+
while chatting:
138+
chatting = await chat(kernel, chat_func, context)
139+
140+
141+
if __name__ == "__main__":
142+
asyncio.run(main())

‎python/samples/kernel-syntax-examples/google_palm_chat_with_skill.py

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
5+
import semantic_kernel as sk
6+
import semantic_kernel.connectors.ai.google_palm as sk_gp
7+
8+
"""
9+
System messages prime the assistant with different personalities or behaviors.
10+
The system message is added to the prompt template, and a chat history can be
11+
added as well to provide further context.
12+
A system message can only be used once at the start of the conversation, and
13+
conversation history persists with the instance of GooglePalmChatCompletion. To
14+
overwrite the system message and start a new conversation, you must create a new
15+
instance of GooglePalmChatCompletion.
16+
Sometimes, PaLM struggles to use the information in the prompt template. In this
17+
case, it is recommended to experiment with the messages in the prompt template
18+
or ask different questions.
19+
"""
20+
21+
system_message = """
22+
You are a chat bot. Your name is Blackbeard
23+
and you speak in the style of a swashbuckling
24+
pirate. You reply with brief, to-the-point answers
25+
with no elaboration. Your full name is Captain
26+
Bartholomew "Blackbeard" Thorne.
27+
"""
28+
29+
kernel = sk.Kernel()
30+
api_key = sk.google_palm_settings_from_dot_env()
31+
palm_chat_completion = sk_gp.GooglePalmChatCompletion("models/chat-bison-001", api_key)
32+
kernel.add_chat_service("models/chat-bison-001", palm_chat_completion)
33+
prompt_config = sk.PromptTemplateConfig.from_completion_parameters(
34+
max_tokens=2000, temperature=0.7, top_p=0.8
35+
)
36+
prompt_template = sk.ChatPromptTemplate(
37+
"{{$user_input}}", kernel.prompt_template_engine, prompt_config
38+
)
39+
prompt_template.add_system_message(system_message) # Add the system message for context
40+
prompt_template.add_user_message(
41+
"Hi there, my name is Andrea, who are you?"
42+
) # Include a chat history
43+
prompt_template.add_assistant_message("I am Blackbeard.")
44+
function_config = sk.SemanticFunctionConfig(prompt_config, prompt_template)
45+
chat_function = kernel.register_semantic_function(
46+
"PirateSkill", "Chat", function_config
47+
)
48+
49+
50+
async def chat() -> bool:
51+
context_vars = sk.ContextVariables()
52+
53+
try:
54+
user_input = input("User:> ")
55+
context_vars["user_input"] = user_input
56+
except KeyboardInterrupt:
57+
print("\n\nExiting chat...")
58+
return False
59+
except EOFError:
60+
print("\n\nExiting chat...")
61+
return False
62+
63+
if user_input == "exit":
64+
print("\n\nExiting chat...")
65+
return False
66+
67+
answer = await kernel.run_async(chat_function, input_vars=context_vars)
68+
print(f"Blackbeard:> {answer}")
69+
return True
70+
71+
72+
async def main() -> None:
73+
chatting = True
74+
while chatting:
75+
chatting = await chat()
76+
77+
78+
if __name__ == "__main__":
79+
asyncio.run(main())

‎python/semantic_kernel/connectors/ai/google_palm/__init__.py

+11-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,17 @@
11
# Copyright (c) Microsoft. All rights reserved.
22

3+
from semantic_kernel.connectors.ai.google_palm.services.gp_chat_completion import (
4+
GooglePalmChatCompletion,
5+
)
36
from semantic_kernel.connectors.ai.google_palm.services.gp_text_completion import (
47
GooglePalmTextCompletion,
58
)
9+
from semantic_kernel.connectors.ai.google_palm.services.gp_text_embedding import (
10+
GooglePalmTextEmbedding,
11+
)
612

7-
__all__ = ["GooglePalmTextCompletion"]
13+
__all__ = [
14+
"GooglePalmTextCompletion",
15+
"GooglePalmChatCompletion",
16+
"GooglePalmTextEmbedding",
17+
]

0 commit comments

Comments
 (0)