3 Star 8 Fork 3

百度开源 / unit-uskit

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

USKit

简介

在真实的对话系统中,通常会要求同时支持多种对话技能,比如育儿对话机器人中要求同时支持音乐、故事、古诗、百科等技能。这涉及到多个技能的管理、召回、排序和选择等策略,在具体的实现中,通常由一个统一调度对话中控 US (Unified Scheduler) 来提供这些能力。

USKit 作为 UNIT 的开源对话中控模块,主要负责多个对话技能的接入、调度、排序、选择、回复生成以及多轮对话 session 的管理,无缝对接 UNIT 平台能力,可以方便开发者通过配置快速搭建和定制符合自己业务场景的对话中控,将多种技能整合为统一的对话机器人为用户提供服务。

主要功能

核心概念

在开始之前,我们先定义以下两个概念:

  1. 对话技能 (bot skill):指某个特定场景下的对话能力,一个对话技能对应一个具体的对话场景,例如天气技能负责提供天气场景下的对话能力。
  2. 对话机器人 (bot service):由多个对话技能整合而成的统一对话服务,可以同时支持多个对话场景的人机对话,一个对话机器人对应一个具体业务的整体解决方案,比如车载对话机器人、育儿对话机器人等。USKit 提供的正是快速搭建对话机器人的能力。

功能介绍

USKit 作为 UNIT 的开源中控模块,通过配置驱动和内置表达式运算支持,提供了以下能力:

  • 支持配置表达式运算,内置常用函数,提供丰富的表达能力,通过修改配置即可轻松实现策略的定制
  • 支持定义后端服务的请求构造和接入策略 (后端服务泛指各种通过网络访问的远程服务,比如 UNIT 技能、DMKit 服务、Redis session 服务等),支持的通信协议包括 HTTPRedis
  • 支持定义后端服务结果的抽取 (extract) 和变换 (transform) 策略
  • 支持定义多种对话技能的排序 (ranking) 和选择策略
  • 支持定义对话中控流程 (chatflow) 策略
  • 内置 UNIT 技能协议请求构造和结果解析策略的配置支持,只需配置技能 ID 即可实现对话技能的快速接入

架构

USKit 针对对话中控的能力做了抽象,抽取出不同对话场景中下中控的通用能力,避免不必要的重复开发。同时按照配置化驱动和高扩展性的思想进行整体架构的设计,方便开发者通过配置快速构建和定制适用于特定业务场景的对话中控。USKit的整体架构如下图:

USKit整体架构

系统主要由以下4个核心部分组成:

  • 表达式引擎:负责表达式的解析和执行求值,详细的运算支持可以参见配置表达式运算支持,表达式引擎为下面三个引擎提供了基础支持,基于表达式引擎,可以实现根据用户请求动态生成配置
  • 后端服务管理引擎:负责后端服务的接入、请求构造和结果解析抽取等策略的管理,通过 backend.conf 进行策略的配置
  • 排序策略管理引擎:负责技能的排序规则的管理,用于多技能的排序,通过 rank.conf 进行策略的配置
  • 对话流程管理引擎:负责对话中控流程的策略的管理,用于定义机器人中控的逻辑执行流程:包括调用后端服务引擎召回技能 (recall)、调用排序引擎进行技能排序 (rank)、结果选择和输出等,通过 flow.conf 进行策略的配置

开发者在使用 USKit 的时候,只需配置上述3个配置文件,即可完成对话中控的搭建,后续策略的变动和升级也通过修改配置文件即可完成,可以让开发者关注对话机器人中控本身的策略逻辑,不需要重复开发框架代码

3个配置文件的详细说明可以参考详细配置说明

安装

从 GitHub 获取 USKit 的源代码:

git clone https://github.com/baidu/unit-uskit.git
cd unit-uskit

USKit 提供了两种安装方式:使用 Docker 构建镜像和手动安装。推荐使用 Docker 进行安装。

Docker 安装

docker build -t unit-uskit .

我们也提供成功编译的 Docker 镜像,可使用如下命令拉取镜像:

docker pull docker.io/shanetian/unit-uskit:v3.0.0

手动安装

1. 安装依赖

USKit 基于以下工具包进行开发:

  • brpc
  • boost
  • protobuf
  • flex >= 2.6.4
  • bison >= 3.0.4
  • cmake >= 2.8.10

目前支持 Ubuntu,CentOS 等系统环境,Ubuntu 推荐使用 16.04 或以上版本,CentOS 推荐使用 7 以上版本,编译 USKit 前请确保对应版本的依赖已经正确安装:

Ubuntu
sh deps.sh ubuntu
CentOS
sh deps.sh centos

2. 编译USKit

mkdir _build && cd _build && cmake .. && make

注:编译前请检查 gcc 版本,若版本较低不支持 C++14,则使用 source /opt/rh/devtoolset-7/enable 切换到新安装的 gcc 再编译即可。

如果希望打印更多 Debug 信息,可以在生成执行 cmake 时指定为 Debug,在运行 USKit 的过程中将会打出更详细的 log 信息,方便问题定位:

mkdir _build && cd _build && cmake -DCMAKE_BUILD_TYPE=Debug .. && make

注:打开该选项会影响性能,建议只在开发环境中使用。

USKit 代码目录结构

.
├── CMakeLists.txt
├── README.md
├── _build              # 编译产出,在该目录下修改配置文件以及运行 ./uskit 启动服务
│   ├── conf            # 配置目录
│   │   ├── us          # 对话机器人的配置,每个子目录对应一个具体业务场景下的对话机器人
│   │   │   └── demo    # USKit 教程示例的配置及配置生成工具
│   │   │       ├── options.py           # demo 场景里需要开发者自行配置的选项,如技能 ID,API Key 等
│   │   │       ├── conf_generator.py    # 根据 options.py 里声明的配置,自动生成所需配置文件
│   │   │       └── conf_templates       # 配置模板目录
│   │   ├── gflags.conf                  # USKit 启动参数默认参数配置
│   │   └── us.conf                      # USKit 本身相关的配置,用于指定加载的对话机器人
│   └── uskit           # USKit 主程序
├── conf                # 配置目录,编译成功后会被复制到 _build/conf
├── docs                # 详细文档
├── proto               # protobuf 文件
├── src                 # 源代码
└── third_party         # 第三方依赖目录

使用

USKit 主要通过配置来驱动中控的搭建和定制,使用方只需要配置对应目录下的 backend.confrank.conf 以及 flow.conf,然后重启服务即可生效。

新建对话中控

在 USKit 中,新建一个对话场景下的机器人中控,只需在 _build/conf/us 目录下新增对应的目录,例如,新增一个对话机器人 demo_service

mkdir -p _build/conf/us/demo_service

在该目录下,编写对应的配置文件:

  • backend.conf:指定需要接入的资源对应的请求构造和结果解析配置
  • rank.conf:召回结果的排序配置
  • flow.conf:中控整体对话流程的配置

注册对话中控

同时修改 us.conf,注册需要加载的对话机器人,新添如下一行:

load : "demo_service"

启动服务

进入 _build 目录运行 USKit:

./uskit

命令参数说明:

  • --port:指定 USKit 服务的端口,默认为 8888
  • --idle_timeout_s:指定 client 多少秒没有读/写操作即关闭链接,默认为 -1,即不关闭
  • --us_conf:指定 us.conf 的路径,默认为 ./conf/us.conf
  • --url_path:指定 USKit 服务的 url 路径,默认为 /us
  • --http_verbose: 在 stderr 输出 http 网络请求和返回的数据
  • --http_verbose_max_body_length: 指定 http_verbose 输出数据的最大长度
  • --redis_verbose:在 stderr 输出 redis 请求和返回的数据

成功启动 USKit 服务后,可以通过 <HOST>:8888/us 发起 HTTP POST 请求,请求体使用 json 格式,请求参数如下:

  • usid:指定需要请求的对话机器人中控 id,比如上文中创建的 demo_service
  • logid:每个请求对应 id,用于将上下游串起来的唯一 id,方便问题追踪定位
  • query:用户请求的 query
  • uuid:用户对应的 id,用于唯一区分一个用户

样例:

{"usid": "demo_service", "logid": "123456", "query": "北京今天天气怎么样", "uuid": "123"}

返回结果同样采用 json 格式,包含以下三个字段:

  • error_code:错误码
  • error_msg:具体错误信息
  • result:通过对话中控策略执行得到的请求响应

样例:

{"error_code": 0, "error_msg": "OK", "result": "好的"}

更多文档

FAQ

  1. 执行 deps.sh 的过程中 clone boost 失败导致后续依赖安装失败

    由于 boost 库较大,GitHub 访问较慢导致下载失败。可以在 boost 官网手动下载包至 third_party/ 并编译:

    tar -zxf boost_1_76_0.tar.gz && mv boost_1_76_0 boost
    cd boost
    mkdir -p _build/output
    ./bootstrap.sh --prefix=./_build/output
    ./b2 install

如何贡献

  • 欢迎提交任何问题到 Github Issues,可以是使用过程中遇到的问题、bug reporting 或者新需求的讨论。
  • 对于 issues 中的问题欢迎贡献并发起 pull request。
  • 定制新的自定义函数和策略,如果足够通用,可以提交 pull request 合入作为内置。

讨论

  • issue 发起问题讨论,如果是问题选择类型为问题即可。
  • 欢迎加入 UNIT QQ群(584835350)交流讨论。

Copyright and License

USKit is licensed under the Apache License, Version 2.0

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

USKit作为UNIT的开源对话中控模块,主要负责多个对话技能的接入、调度、排序、选择、回复生成以及多轮对话session的管理,无缝对接UNIT平台能力,可以方便开发者通过配置快速搭建和定制符合自己业务场景的对话中控,将多种技能整合为统一的对话机器人为用户提供服务。 展开 收起
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
1
https://gitee.com/baidu/unit-uskit.git
git@gitee.com:baidu/unit-uskit.git
baidu
unit-uskit
unit-uskit
master

搜索帮助