Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

To select a json object from mongodb in clickhouse. #64097

Open
HitendraSinghRathore opened this issue May 19, 2024 · 4 comments
Open

To select a json object from mongodb in clickhouse. #64097

HitendraSinghRathore opened this issue May 19, 2024 · 4 comments
Labels
bug Confirmed user-visible misbehaviour in official release question Question?

Comments

@HitendraSinghRathore
Copy link

HitendraSinghRathore commented May 19, 2024

I have a nested object in my mongo db table the column is
affiliate_bag_details

{
"test": "12",
"obj": {
 "nested": [1,2,4]
}
}

while fetching the same using the function MongoDB

SELECT 
    bag_id, 
    display_name, 
    affiliate_bag_details
FROM mongodb(
    '<IP>:host', 
    'db', 
    'collection',
    'username',
    'password!2018',
    'affiliate_bag_details String',
    'connectTimeoutMS=10000'
) 
LIMIT 10;

Getting the following error
Exception: Type mismatch, expected String, got type id = 3 for column affiliate_bag_details: While executing MongoDB.
Can anyone please suggest what is an alternative schema that i can use to perform the above operation?

@HitendraSinghRathore HitendraSinghRathore added the question Question? label May 19, 2024
@HitendraSinghRathore
Copy link
Author

HitendraSinghRathore commented May 20, 2024

Tried using

SELECT 
    bag_id, 
    display_name, 
    affiliate_bag_details.test
FROM mongodb(
    '<IP>:host', 
    'db', 
    'collection',
    'username',
    'password!2018',
    'affiliate_bag_details Nested( test String)',
    'connectTimeoutMS=10000'
) 
LIMIT 10;

This does not throw error but date comes as [] event tho source table has values present.

@kssenii
Copy link
Member

kssenii commented May 21, 2024

Looks like an easy fixable bug: we just need to support struct ElementTraits<Document::Ptr> in

case ValueType::vtString:
{
if (value.type() == Poco::MongoDB::ElementTraits<ObjectId::Ptr>::TypeId)
{
std::string string_id = value.toString();
assert_cast<ColumnString &>(column).insertData(string_id.data(), string_id.size());
break;
}
else if (value.type() == Poco::MongoDB::ElementTraits<String>::TypeId)
{
String string = static_cast<const Poco::MongoDB::ConcreteElement<String> &>(value).value();
assert_cast<ColumnString &>(column).insertData(string.data(), string.size());
break;
}

@kssenii kssenii added the bug Confirmed user-visible misbehaviour in official release label May 21, 2024
@HitendraSinghRathore
Copy link
Author

Will it be able to read JSON objects from mongoDB as is (by mentioning Nested or JSON in schema), or will it still need to read it as String then cast post read?

@jakczer
Copy link

jakczer commented May 24, 2024

Hi @kssenii will it be fixed soon?
maybe is it possible to add this hotfix to next release?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Confirmed user-visible misbehaviour in official release question Question?
Projects
None yet
Development

No branches or pull requests

3 participants