BugFix :: JSON Flatten is unable to handle a list of json objects #47
Conversation
✅ Deploy Preview for redpanda-labs-preview ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
|
||
| default: | ||
| return isNodeFlattened |
There was a problem hiding this comment.
Why are we returning early here? Does this end up skipping over elements if there are heterogeneous lists? I.E.
[
"foo",
{"a":4},
5,
]I think the above case will drop the 5 value.
There was a problem hiding this comment.
The above doesn't drop values, but it doesn't flatten the {"a": 4} Object, either. If the first element ("foo") is removed, it will drop the 5.
I'm conflicted on what the correct approach should be here. My original intent was to provide a close to drop in replacement for the Kafka Connect Flatten SMT. I believe the Flatten SMT supports heterogeneous Array flattening, but requires a schema. Maybe that's the correct approach here?
There was a problem hiding this comment.
Sorry, I didn't have much time to put in this PR lately. It has been a moment since I opened this PR, I don't remember why, to be honest. I will look it up as well when I will have a moment.
Hello!
It is me again!
I noticed the example couldn't handle lists of JSON objects as well. I fixed it, and I made quite a mess last time I contributed, so I cleaned behind myself 😉 .
As you will be able to see in my example below, I went for a common way to annotate the element that is part of a list by identifying it with the index of the element of the list.
Example:
Before:
{ "id": 1234, "content": { "id": 123, "name": { "first": "Dave", "middle": null }, "addresses": [ { "street":"someStreet", "appartments":[] }, { "street":"someOtherStreet", "appartments":[ 1, 2, 3, 4 ] } ] }, "data": [1, "fish", 2, "fish"], "more_data": { "id": 123, "empty": {}, "name": { "first": "Bob", "middle": "Jr" } }, "content": "test", "empty_again": {} }After:
{ "id": 1234, "content.id": 123, "content.name.first": "Dave", "content.name.middle": null, "content.addresses.0.street": "someStreet", "content.addresses.0.appartments": [], "content.addresses.1.street": "someOtherStreet", "content.addresses.1.appartments": [1, 2, 3, 4], "data": [1, "fish", 2, "fish"], "more_data.id": 123, "more_data.empty": {}, "more_data.name.first": "Bob", "more_data.name.middle": "Jr", "content": "test", "empty_again": {} }Thank you 😄