by Wsnhg

轻松玩转书生·浦语大模型趣味Demo_哔哩哔哩_bilibili

• InternLM-Chat-7B 智能对话 Demo

  • code

    import torch
    from transformers import AutoTokenizer, AutoModelForCausalLM
    
    model_name_or_path = "/root/model/Shanghai_AI_Laboratory/internlm-chat-7b"
    
    tokenizer = AutoTokenizer.from_pretrained(
        model_name_or_path,
        trust_remote_code=True，
    )
    model = AutoModelForCausalLM.from_pretrained(
        model_name_or_path,
        trust_remote_code=True,
        torch_dtype=torch.bfloat16, 
        device_map='auto'
    )
    model = model.eval()
    
    system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
    - InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
    - InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
    """
    
    messages = [(system_prompt, '')]
    
    print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")
    
    while True:
        input_text = input("User  >>> ")
        input_text = input_text.replace(' ', '')
        if input_text == "exit":
            break
        response, history = model.chat(tokenizer, input_text, history=messages)
        messages.append((input_text, response))
        print(f"robot >>> {response}")
    

  • 笔记

    • 可以看出internLM的接口是适配hugging face的，可以直接使用hugging face的transformers包来进行前向推理。
    • 使用transformers.AutoTokenizer来实例化tokenizer
    • 使用transformers.AutoModelForCausalLM来实例化model
    • 通过model.chat来实现对话，并且可以使用history=messages来注入记录的历史对话信息

• Lagent 智能体工具调用 Demo

  • code

  import copy
  import os
  
  import streamlit as st
  from streamlit.logger import get_logger
  
  from lagent.actions import ActionExecutor, GoogleSearch, PythonInterpreter
  from lagent.agents.react import ReAct
  from lagent.llms import GPTAPI
  from lagent.llms.huggingface import HFTransformerCasualLM
  
  class SessionState:
  
      def init_state(self):
          """Initialize session state variables."""
          st.session_state['assistant'] = []
          st.session_state['user'] = []
  
          #action_list = [PythonInterpreter(), GoogleSearch()]
          action_list = [PythonInterpreter()]
          st.session_state['plugin_map'] = {
              action.name: action
              for action in action_list
          }
          st.session_state['model_map'] = {}
          st.session_state['model_selected'] = None
          st.session_state['plugin_actions'] = set()
  
      def clear_state(self):
          """Clear the existing session state."""
          st.session_state['assistant'] = []
          st.session_state['user'] = []
          st.session_state['model_selected'] = None
          if 'chatbot' in st.session_state:
              st.session_state['chatbot']._session_history = []
  
  ......  # 代码太长可以在上面的 <https://github.com/InternLM/tutorial/blob/main/helloworld/hello_world.md> 中找到
  
  

  • 笔记
    • 可以看出lagent的使用也是类似的形式，通过SessionState来记录设定，然后通过st.session_state['chatbot'].chat(user_input)来获取模型输出

• 浦语·灵笔图文理解创作 Demo

  • 笔记
    • 目前只是一个demo，暂时没有有营养的内容，后续读了代码再更新