Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

paimon catalog 不支持的字段时表不能查询 #45812

Open
Oloong opened this issue May 17, 2024 · 0 comments
Open

paimon catalog 不支持的字段时表不能查询 #45812

Oloong opened this issue May 17, 2024 · 0 comments
Labels
type/bug Something isn't working

Comments

@Oloong
Copy link

Oloong commented May 17, 2024

Steps to reproduce the behavior (Required)

MYSQL源表:
CREATE TABLE dinky_user (
id int(11) NOT NULL AUTO_INCREMENT COMMENT 'ID',
username varchar(50) NOT NULL COMMENT 'username',
user_type int(11) DEFAULT '1',
password varchar(50) DEFAULT NULL COMMENT 'password',
nickname varchar(50) DEFAULT NULL COMMENT 'nickname',
worknum varchar(50) DEFAULT NULL COMMENT 'worknum',
avatar blob DEFAULT NULL COMMENT 'avatar',
mobile varchar(20) DEFAULT NULL COMMENT 'mobile phone',
enabled tinyint(1) NOT NULL DEFAULT '1' COMMENT 'is enable',
super_admin_flag tinyint(4) DEFAULT '0' COMMENT 'is super admin(0:false,1true)',
is_delete tinyint(1) NOT NULL DEFAULT '0' COMMENT 'is delete',
create_time datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'create time',
update_time datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'update time',
PRIMARY KEY (id) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=14 DEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC COMMENT='user';
源MYSQL表字段是 blob 经FLINKCDC 到paimon表 转换为byte

drop CATALOG mypaimon;
CREATE EXTERNAL CATALOG mypaimon
comment "外部catalog数据湖paimon"
PROPERTIES ("paimon.catalog.type" = "filesystem",
"type" = "paimon",
"paimon.catalog.warehouse" = "hdfs://bd/data/paimon"
);
set catalog mypaimon;
desc dinky.dinky_user;
image
-- 查全部 报错
select * from dinky.dinky_user

1064 - Getting analyzing error. Detail message: Datatype of external table column [avatar] is not supported!.
时间: 0.008s

-- 无数据时 不查问题列 正常
select id,username from dinky.dinky_user
-- 有数据时 不查问题列 依然报错
select id,username from dinky.dinky_user

1064 - java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index 13 out of bounds for length 13
时间: 0.081s
1064 - java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException
时间: 0.072s
select count(*) from dinky.dinky_user;
1064 - java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index 13 out of bounds for length 13
时间: 0.081s
1064 - java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException
时间: 0.072s
上面两种报错信息随机出现

调整好字段(源表调整成varchar),依然报错!!!
set catalog mypaimon;
desc dinky.dinky_user;
image

无数据时,查询正常

有数据时,同样报错!!!
select * from dinky.dinky_user

1064 - java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index 13 out of bounds for length 13
时间: 0.064s

敢情PAIMON表都查不了???

Expected behavior (Required)

至少不在不涉及到未识别字段时不报错
建议对于不识别的字段统一转成STRING类型,或做个配置让用户自行决定是否兼容

Real behavior (Required)

报错,整个表都废了

StarRocks version (Required)

v3.1.11

@Oloong Oloong added the type/bug Something isn't working label May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant