[v2] Update JSON formatter to encode bytes as Base64 strings

[aemous]

Follow-up bugfix for #9388.

Current Problem

When a service sends back a response containing a member with raw bytes, the CLI JSON formatter fails with an error and does not output the complete response. Since CBOR is not a modeled protocol by any service (as of writing), this bugfix should not be a breaking change.

Explanation of problem

When using the JSON wire protocol, a service encodes the raw bytes as base64 (string) to represent it in JSON before sending it back. CLI's JSON formatter perfectly handles strings.
However, when using CBOR as the wire protocol, the service sends the raw bytes, so now it is up to us to encode the bytes as base64 to represent it in JSON format.

Description of changes

• Modifies JSON formatter so that it serializes bytes into a UTF-8 Base64 string.
• Add a regression unit test to verify that binary data in the response gets base-64 encoded.

Description of tests

• Call a service operation using the CBOR protocol and receive a response containing a member with raw bytes. Prior to this change, error code 255 is observed and the full response is not printed. After the change, the output is fully printed, and it matches the output seen when using the JSON protocol on the same operation.
• Ran and passed all test suites and CI.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[aemous]

For reviewer, thoughts on adding a regression test that would catch this? I'm thinking maybe in /tests/functional/test_output.py, mock a service response that contains raw bytes, and assert that the JSON formatter outputs the base64 encoded string.

[aemous]

For reviewer, I chose utf-8 here because this is a JSON encoder, and according to the JSON spec, JSON text SHALL be encoded in Unicode. The default encoding is UTF-8.

[aemous]

For reviewer, according to Python documentation for json.dump, JSON encoders should be raising a TypeError for any type that it does not encode. This will prevent the ambiguous "Circular reference detected" if the issue recurs in the future with a different type.

[ashovlin]

1. Could you add a test case exercising this?
2. Are the other output formats (table, text, yaml) okay?

[aemous]

1. Could you add a test case exercising this?
2. Are the other output formats (table, text, yaml) okay?

1. Will address in next revision
2. (i) YAML supports raw bytes out-of-the-box by encoding it into a base64 string (ii) for text format, it prints the bytes exactly how Python prints bytes (e.g. b'\xac\x09'). (iii) table format similar to text, except there's table ASCII art around it.

The reason the issue addressed is unique to JSON is because JSON is our only formatter that does not support binary data.