ChatGLM2模型推理
原先的配置文件中,参数pre_seq_len: None已经设置为None,但是推理的时候貌似会有问题。
以下代码不会出错:
model = AutoModel.from_pretrained("glm2_6b", use_past=False, seq_length=512, pre_seq_len=None)
以下代码会出错:
model = AutoModel.from_pretrained("glm2_6b", use_past=False, seq_length=512)
报错如下:
mindformers/mindformers/models/glm2/glm2.py:92, in ChatGLM2Model.init(self, config, **kwargs)
85 self.casual_mask = LowerTriangularMaskWithDynamic(seq_length=config.seq_length,
86 compute_type=config.compute_dtype,
87 mask_type=config.mask_type,
88 is_dynamic=config.is_dynamic,
89 pad_token_id=config.pad_token_id,
90 use_flash_attention=use_flash_attention_flag)
---> 92 max_seq_length = config.seq_length if not self.pre_seq_len else config.seq_length + self.pre_seq_len
93 self.kvcache_preprocess = KVCachePreprocess(max_batch_size=config.batch_size,
94 max_seq_length=max_seq_length,
95 is_dynamic=config.is_dynamic,
96 use_kvcache_op=config.use_kvcache_op,
97 is_flexible_shape=config.is_flexible_shape,
98 use_paged_attention=self.use_paged_attention)
TypeError: unsupported operand type(s) for +: 'int' and 'str'