What's new in José v8?
Part 2: JSON Parsing and Modification
August 8, 2017
fedora
fedora-security
Wait! What’s José?
José is a general purpose cryptography toolkit which uses the data formats standardized by the JOSE IETF Working Group. By analogy, José is to JOSE what GPG is to OpenPGP and OpenSSL is to X.509.
José provides both a C-language library and a command line interface and is
licensed under the Apache Software License version 2.0. José v8+ is available
on Fedora 26+ (dnf install jose
), Red Hat Enterpirse Linux 7.4+ (yum install jose
) and macOS/Homebrew (brew install jose
).
Motivation
While building José and projects that use it, we kept running into the problem
that we needed to do manipulation of JSON data on the command line. First, we
attempted to use jq. However, we found the interface to be cumbersome.
Our biggest pain point was that we often wanted simple error reporting, but
jq
required us to write complicated error handlers. We also anticipated that
many other consumers of José would have similar needs and we wanted to provide
an integrated solution.
The end result of this development need is the jose fmt
command; a simple
JSON stack machine. We believe that this implementation is both simpler to use
and less error prone. Let’s look at how to use jose fmt
.
JSON Parsing
In our first example, we will extract a single JSON string value from a JSON object:
$ echo '{"key":"value"}' | jose fmt -j- -g key -u-
value
$ echo $?
0
This is simple enough. We pass three instructions to jose fmt
: -j
, -g
and
-u
. First, the -j-
instruction tells jose fmt
to parse JSON from standard
input. The result of this operation is placed on the top of the internal stack.
Second, the -g key
instruction tells jose fmt
to get the value of the
"key"
property. Like before, the result of this operation is placed on the
top of the internal stack. Finally, the -u-
instruction tells jose fmt
to
print the string on the top of the stack to standard output without quotes.
What happens if the input is malformed or we pass instructions that are
invalid? In this case, jose fmt
errors and the return code of the process
is the number of the instruction that failed. For example:
$ jose fmt -j '{}' -g key -u-
$ echo $?
2
In this case, the input is valid JSON, so the first instruction (-j '{}'
)
succeeds. However, the resulting value does not contain the property "key"
,
so the second instruction (-g key
) fails. Thus, the return code is 2
.
JSON Modification
Let’s look at a more complex case. This time we have a nested JSON structure
and we want to change the value of "c"
to 3
. Here’s how we do it:
$ jose fmt -j '{"a":{"b":{"c":7}}}' -g a -g b -j 3 -s c -UUUo-
{"a":{"b":{"c":3}}}
First, we parse the input (-j ...
) then we successively place the nested
objects on the stack (-g a
and -g b
). Next, we place our new value on the
stack (-j 3
) and set it (-s c
), overwriting the previous value. Finally,
we unwind the stack back to the root object (-UUU
) and output the JSON to
standard output (-o-
).
URL-Safe Base64
Because we wanted to provide an integrated solution, jose fmt
also provides
native support for performing URL-Safe Base64 encoding and decoding. This makes
it extremely easy to parse objects such as JWE and JWS. For example, imagine
we have a JWE that looks something like this:
$ cat my.jwe
{"protected":"eyJlbmMiOiJBMjU2Q0JDLUhTNTEyIn0",...}
If we wanted to extract a value from the JWE Protected Header using jq
, we’d
have to extract the "protected"
property, then decode it and, finally, parse
it again. With jose fmt
we can do this in all one step:
$ jose fmt -j my.jwe -g protected -y -o-
{"enc":"A256CBC-HS512"}
$ jose fmt -j my.jwe -g protected -y -g enc -u-
A256CBC-HS512
In this example we use the -y
instruction to decode the URL-Safe Base64 and
parse the resulting bytes as JSON. If this succeeds, the JSON value produced
from this process pushed onto the top of the stack (just like all other
operations). From here, it can be manipulated just like any other value.
Conclusion
The jose fmt
is a flexible utility that can be used to parse JSON of all
sorts. You can use it to read values from JSON. You can use it to transform
JSON values to new forms. You can use it to modify values within nested JSON
objects. You can also use it to build up JSON values gradually. Try it in your
project today!
For a detailed list of all the possible instructions, see jose fmt -h
.