Object validation torture (AOR proposal)

[codalogic]

I've been thinking about validation of objects. If we have the following JCR:

{ "bar":string, ( "foo":integer | "baz":string ) }

should the following JSON be valid or not:

{ "bar":"thing", "foo":2, "baz": "thingy" }

My feeling is not because the author is saying they want one or the other.

What do others expect?

[johnwcowan]

I agree with you. If you want coffee or tea, it doesn't mean you want one of each.

"If this is coffee, bring me tea; but if this is tea, bring me coffee."

[anewton1998]

I know this seems weird, but in the spirit of JSON (and sometimes XML depending on opinion) all members not expressly forbidden are allowed. Or put a different way, unspecified members of an object are ignored. If you want that to work, you have to explicitly "close off" or "restrict" the object.

Here's what you are doing:

→  jcr -v -R '{ "bar":string, ( "foo":integer | "baz":string ) }' -J '{ "bar":"thing", "foo":2, "baz": "thingy" }'
"Ruleset Parse Tree"
[{:object_rule=>
   [{:member_rule=>
      {:member_name=>{:q_string=>"bar"@3},
       :primitive_rule=>{:string=>"string"@8}}},
    {:sequence_combiner=>","@14,
     :group_rule=>
      [{:member_rule=>
         {:member_name=>{:q_string=>"foo"@19},
          :primitive_rule=>{:integer_v=>"integer"@24}}},
       {:choice_combiner=>"|"@32,
        :member_rule=>
         {:member_name=>{:q_string=>"baz"@35},
          :primitive_rule=>{:string=>"string"@40}}}]}]}]
"Ruleset Map"
"Evaluating Root:"
" { \"bar\" : string ,  ( \"foo\" : integer | \"baz\" : string ) }"
[ 1:[1, 4]@3 ] Evaluating object rule starting at '"bar"@3' ( line 1 column 4 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 1:[1, 4]@3 ] object definition:  { "bar" : string ,  ( "foo" : integer | "baz" : string ) } data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 4]@3 ] Evaluating member rule for key 'bar' starting at '"bar"@3' ( line 1 column 4 ) against  data: "thing"
[ 2:[1, 4]@3 ] member definition: "bar" : string data: ["bar", "thing"]
[ 3:[1, 9]@8 ] Evaluating value rule starting at '"string"@8' ( line 1 column 9 )
[ 3:[1, 9]@8 ] value definition: string data: "thing"
[ 3:[1, 9]@8 ] Value evaluation is true
[ 2:[1, 4]@3 ] Member evaluation is true
[ 1:[1, 4]@3 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 20]@19 ] Evaluating object rule starting at '"foo"@19' ( line 1 column 20 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 2:[1, 20]@19 ] object definition:  { "foo" : integer | "baz" : string } data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 2:[1, 20]@19 ] rule repetition min = 1 max = 1 repetition step =
[ 3:[1, 20]@19 ] Evaluating member rule for key 'foo' starting at '"foo"@19' ( line 1 column 20 ) against  data: 2
[ 3:[1, 20]@19 ] member definition: "foo" : integer data: ["foo", 2]
[ 4:[1, 25]@24 ] Evaluating value rule starting at '"integer"@24' ( line 1 column 25 )
[ 4:[1, 25]@24 ] value definition: integer data: 2
[ 4:[1, 25]@24 ] Value evaluation is true
[ 3:[1, 20]@19 ] Member evaluation is true
[ 2:[1, 20]@19 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 2:[1, 20]@19 ] Object evaluation is true
[ 1:[1, 4]@3 ] Object evaluation is true
Success!

But here it is with @{not} //:any @+ to close off the object:

→  jcr -v -R '{ "bar":string, ( "foo":integer | "baz":string ), @{not} //:any @+ }' -J '{ "bar":"thing", "foo":2, "baz": "thingy" }'
"Ruleset Parse Tree"
[{:object_rule=>
   [{:member_rule=>
      {:member_name=>{:q_string=>"bar"@3},
       :primitive_rule=>{:string=>"string"@8}}},
    {:sequence_combiner=>","@14,
     :group_rule=>
      [{:member_rule=>
         {:member_name=>{:q_string=>"foo"@19},
          :primitive_rule=>{:integer_v=>"integer"@24}}},
       {:choice_combiner=>"|"@32,
        :member_rule=>
         {:member_name=>{:q_string=>"baz"@35},
          :primitive_rule=>{:string=>"string"@40}}}]},
    {:sequence_combiner=>","@48,
     :member_rule=>
      [{:not_annotation=>"not"@52},
       {:member_regex=>{:regex=>[], :regex_modifiers=>[]}},
       {:primitive_rule=>{:any=>"any"@60}}],
     :one_or_more=>"+"@65}]}]
"Ruleset Map"
"Evaluating Root:"
" { \"bar\" : string ,  ( \"foo\" : integer | \"baz\" : string ) ,  @{not} /[]/ : any }"
[ 1:[1, 4]@3 ] Evaluating object rule starting at '"bar"@3' ( line 1 column 4 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 1:[1, 4]@3 ] object definition:  { "bar" : string ,  ( "foo" : integer | "baz" : string ) ... data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 4]@3 ] Evaluating member rule for key 'bar' starting at '"bar"@3' ( line 1 column 4 ) against  data: "thing"
[ 2:[1, 4]@3 ] member definition: "bar" : string data: ["bar", "thing"]
[ 3:[1, 9]@8 ] Evaluating value rule starting at '"string"@8' ( line 1 column 9 )
[ 3:[1, 9]@8 ] value definition: string data: "thing"
[ 3:[1, 9]@8 ] Value evaluation is true
[ 2:[1, 4]@3 ] Member evaluation is true
[ 1:[1, 4]@3 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 20]@19 ] Evaluating object rule starting at '"foo"@19' ( line 1 column 20 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 2:[1, 20]@19 ] object definition:  { "foo" : integer | "baz" : string } data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 2:[1, 20]@19 ] rule repetition min = 1 max = 1 repetition step =
[ 3:[1, 20]@19 ] Evaluating member rule for key 'foo' starting at '"foo"@19' ( line 1 column 20 ) against  data: 2
[ 3:[1, 20]@19 ] member definition: "foo" : integer data: ["foo", 2]
[ 4:[1, 25]@24 ] Evaluating value rule starting at '"integer"@24' ( line 1 column 25 )
[ 4:[1, 25]@24 ] value definition: integer data: 2
[ 4:[1, 25]@24 ] Value evaluation is true
[ 3:[1, 20]@19 ] Member evaluation is true
[ 2:[1, 20]@19 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 2:[1, 20]@19 ] Object evaluation is true
[ 1:[1, 4]@3 ] rule repetition min = 1 max = Infinity repetition step =
[ 1:[1, 4]@3 ] Scanning object for [].
[ 2:[1, 53]@52 ] Evaluating member rule for key 'baz' starting at '"not"@52' ( line 1 column 53 ) against  data: "thingy"
[ 2:[1, 53]@52 ] member definition:  @{not} /[]/ : any data: ["baz", "thingy"]
[ 2:[1, 53]@52 ] Noting empty regular expression.
[ 3:[1, 61]@60 ] Evaluating value rule starting at '"any"@60' ( line 1 column 61 )
[ 3:[1, 61]@60 ] value definition: any data: "thingy"
[ 3:[1, 61]@60 ] Value evaluation is true
[ 2:[1, 53]@52 ] Not annotation changing result from true to false
[ 2:[1, 53]@52 ] Member evaluation failed:
[ 2:[1, 53]@52 ] ** LIKELY ROOT CAUSE FOR FAILURE **
[ 2:[1, 53]@52 ] ***********************************
[ 2:[1, 53]@52 ] Failed rule at line,column: [1, 53] file position offset: 52
[ 2:[1, 53]@52 ] member definition:  @{not} /[]/ : any data: ["baz", "thingy"]
[ 2:[1, 53]@52 ] JSON that failed to validate: ["baz","thingy"]
[ 2:[1, 53]@52 ] ***********************************
[ 1:[1, 4]@3 ] Found 0 matching members repetitions in object with min 1 and max Infinity
[ 1:[1, 4]@3 ] Object evaluation failed: object does not contain {:sequence_combiner=>","@48, :member_rule=>[{:not_annotation=>"not"@52}, {:member_regex=>{:regex=>[], :regex_modifiers=>[]}}, {:primitive_rule=>{:any=>"any"@60}}], :one_or_more=>"+"@65} for  rule at '"bar"@3' ( line 1 column 4 ) [ [{:member_rule=>{:member_name=>{:q_string=>"bar"@3}, :primitive_rule=>{:string=>"string"@8}}}, {:sequence_combiner=>","@14, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"foo"@19}, :primitive_rule=>{:integer_v=>"integer"@24}}}, {:choice_combiner=>"|"@32, :member_rule=>{:member_name=>{:q_string=>"baz"@35}, :primitive_rule=>{:string=>"string"@40}}}]}, {:sequence_combiner=>","@48, :member_rule=>[{:not_annotation=>"not"@52}, {:member_regex=>{:regex=>[], :regex_modifiers=>[]}}, {:primitive_rule=>{:any=>"any"@60}}], :one_or_more=>"+"@65}] ] from rule at '"bar"@3' ( line 1 column 4 )
Failure: object does not contain {:sequence_combiner=>","@48, :member_rule=>[{:not_annotation=>"not"@52}, {:member_regex=>{:regex=>[], :regex_modifiers=>[]}}, {:primitive_rule=>{:any=>"any"@60}}], :one_or_more=>"+"@65} for  rule at '"bar"@3' ( line 1 column 4 ) [ [{:member_rule=>{:member_name=>{:q_string=>"bar"@3}, :primitive_rule=>{:string=>"string"@8}}}, {:sequence_combiner=>","@14, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"foo"@19}, :primitive_rule=>{:integer_v=>"integer"@24}}}, {:choice_combiner=>"|"@32, :member_rule=>{:member_name=>{:q_string=>"baz"@35}, :primitive_rule=>{:string=>"string"@40}}}]}, {:sequence_combiner=>","@48, :member_rule=>[{:not_annotation=>"not"@52}, {:member_regex=>{:regex=>[], :regex_modifiers=>[]}}, {:primitive_rule=>{:any=>"any"@60}}], :one_or_more=>"+"@65}] ] from rule at '"bar"@3' ( line 1 column 4 )

If you don't want to close off the object, can you can explicitly change the logic from foo or baz to (foo and not baz) or (baz and not foo). Here is the failure to validate with that.

→  jcr -v -R '{ "bar":string, ( ( "foo":integer , @{not} "baz":string ) | ( "baz":string , @{not} "foo":integer ) ) }' -J '{ "bar":"thing", "foo":2, "baz": "thingy" }'
"Ruleset Parse Tree"
[{:object_rule=>
   [{:member_rule=>
      {:member_name=>{:q_string=>"bar"@3},
       :primitive_rule=>{:string=>"string"@8}}},
    {:sequence_combiner=>","@14,
     :group_rule=>
      [{:group_rule=>
         [{:member_rule=>
            {:member_name=>{:q_string=>"foo"@21},
             :primitive_rule=>{:integer_v=>"integer"@26}}},
          {:sequence_combiner=>","@34,
           :member_rule=>
            [{:not_annotation=>"not"@38},
             {:member_name=>{:q_string=>"baz"@44}},
             {:primitive_rule=>{:string=>"string"@49}}]}]},
       {:choice_combiner=>"|"@58,
        :group_rule=>
         [{:member_rule=>
            {:member_name=>{:q_string=>"baz"@63},
             :primitive_rule=>{:string=>"string"@68}}},
          {:sequence_combiner=>","@75,
           :member_rule=>
            [{:not_annotation=>"not"@79},
             {:member_name=>{:q_string=>"foo"@85}},
             {:primitive_rule=>{:integer_v=>"integer"@90}}]}]}]}]}]
"Ruleset Map"
"Evaluating Root:"
" { \"bar\" : string ,  (  ( \"foo\" : integer ,  @{not} \"baz\" : string ) |  ( \"baz\" : string ,  @{not} \"foo\" : integer ) ) }"
[ 1:[1, 4]@3 ] Evaluating object rule starting at '"bar"@3' ( line 1 column 4 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 1:[1, 4]@3 ] object definition:  { "bar" : string ,  (  ( "foo" : integer ,  @{not} "baz" ... data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 4]@3 ] Evaluating member rule for key 'bar' starting at '"bar"@3' ( line 1 column 4 ) against  data: "thing"
[ 2:[1, 4]@3 ] member definition: "bar" : string data: ["bar", "thing"]
[ 3:[1, 9]@8 ] Evaluating value rule starting at '"string"@8' ( line 1 column 9 )
[ 3:[1, 9]@8 ] value definition: string data: "thing"
[ 3:[1, 9]@8 ] Value evaluation is true
[ 2:[1, 4]@3 ] Member evaluation is true
[ 1:[1, 4]@3 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 22]@21 ] Evaluating object rule starting at '"foo"@21' ( line 1 column 22 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 2:[1, 22]@21 ] object definition:  {  ( "foo" : integer ,  @{not} "baz" : string ) |  ( "ba ... data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 2:[1, 22]@21 ] rule repetition min = 1 max = 1 repetition step =
[ 3:[1, 22]@21 ] Evaluating object rule starting at '"foo"@21' ( line 1 column 22 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 3:[1, 22]@21 ] object definition:  { "foo" : integer ,  @{not} "baz" : string } data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 3:[1, 22]@21 ] rule repetition min = 1 max = 1 repetition step =
[ 4:[1, 22]@21 ] Evaluating member rule for key 'foo' starting at '"foo"@21' ( line 1 column 22 ) against  data: 2
[ 4:[1, 22]@21 ] member definition: "foo" : integer data: ["foo", 2]
[ 5:[1, 27]@26 ] Evaluating value rule starting at '"integer"@26' ( line 1 column 27 )
[ 5:[1, 27]@26 ] value definition: integer data: 2
[ 5:[1, 27]@26 ] Value evaluation is true
[ 4:[1, 22]@21 ] Member evaluation is true
[ 3:[1, 22]@21 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 3:[1, 22]@21 ] rule repetition min = 1 max = 1 repetition step =
[ 4:[1, 39]@38 ] Evaluating member rule for key 'baz' starting at '"not"@38' ( line 1 column 39 ) against  data: "thingy"
[ 4:[1, 39]@38 ] member definition:  @{not} "baz" : string data: ["baz", "thingy"]
[ 5:[1, 50]@49 ] Evaluating value rule starting at '"string"@49' ( line 1 column 50 )
[ 5:[1, 50]@49 ] value definition: string data: "thingy"
[ 5:[1, 50]@49 ] Value evaluation is true
[ 4:[1, 39]@38 ] Not annotation changing result from true to false
[ 4:[1, 39]@38 ] Member evaluation failed:
[ 4:[1, 39]@38 ] ** LIKELY ROOT CAUSE FOR FAILURE **
[ 4:[1, 39]@38 ] ***********************************
[ 4:[1, 39]@38 ] Failed rule at line,column: [1, 39] file position offset: 38
[ 4:[1, 39]@38 ] member definition:  @{not} "baz" : string data: ["baz", "thingy"]
[ 4:[1, 39]@38 ] JSON that failed to validate: ["baz","thingy"]
[ 4:[1, 39]@38 ] ***********************************
[ 3:[1, 22]@21 ] Found 0 matching members repetitions in object with min 1 and max 1
[ 3:[1, 22]@21 ] Object evaluation failed: object does not contain {:sequence_combiner=>","@34, :member_rule=>[{:not_annotation=>"not"@38}, {:member_name=>{:q_string=>"baz"@44}}, {:primitive_rule=>{:string=>"string"@49}}]} for  rule at '"foo"@21' ( line 1 column 22 ) [ [{:member_rule=>{:member_name=>{:q_string=>"foo"@21}, :primitive_rule=>{:integer_v=>"integer"@26}}}, {:sequence_combiner=>","@34, :member_rule=>[{:not_annotation=>"not"@38}, {:member_name=>{:q_string=>"baz"@44}}, {:primitive_rule=>{:string=>"string"@49}}]}] ] from rule at '"bar"@3' ( line 1 column 4 )
[ 2:[1, 22]@21 ] rule repetition min = 1 max = 1 repetition step =
[ 3:[1, 64]@63 ] Evaluating object rule starting at '"baz"@63' ( line 1 column 64 ) against data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 3:[1, 64]@63 ] object definition:  { "baz" : string ,  @{not} "foo" : integer } data: {"bar"=>"thing", "foo"=>2, "baz"=>"thingy"}
[ 3:[1, 64]@63 ] rule repetition min = 1 max = 1 repetition step =
[ 4:[1, 64]@63 ] Evaluating member rule for key 'baz' starting at '"baz"@63' ( line 1 column 64 ) against  data: "thingy"
[ 4:[1, 64]@63 ] member definition: "baz" : string data: ["baz", "thingy"]
[ 5:[1, 69]@68 ] Evaluating value rule starting at '"string"@68' ( line 1 column 69 )
[ 5:[1, 69]@68 ] value definition: string data: "thingy"
[ 5:[1, 69]@68 ] Value evaluation is true
[ 4:[1, 64]@63 ] Member evaluation is true
[ 3:[1, 64]@63 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 3:[1, 64]@63 ] rule repetition min = 1 max = 1 repetition step =
[ 4:[1, 80]@79 ] Evaluating member rule for key 'foo' starting at '"not"@79' ( line 1 column 80 ) against  data: 2
[ 4:[1, 80]@79 ] member definition:  @{not} "foo" : integer data: ["foo", 2]
[ 5:[1, 91]@90 ] Evaluating value rule starting at '"integer"@90' ( line 1 column 91 )
[ 5:[1, 91]@90 ] value definition: integer data: 2
[ 5:[1, 91]@90 ] Value evaluation is true
[ 4:[1, 80]@79 ] Not annotation changing result from true to false
[ 4:[1, 80]@79 ] Member evaluation failed:
[ 3:[1, 64]@63 ] Found 0 matching members repetitions in object with min 1 and max 1
[ 3:[1, 64]@63 ] Object evaluation failed: object does not contain {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]} for  rule at '"baz"@63' ( line 1 column 64 ) [ [{:member_rule=>{:member_name=>{:q_string=>"baz"@63}, :primitive_rule=>{:string=>"string"@68}}}, {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]}] ] from rule at '"bar"@3' ( line 1 column 4 )
[ 2:[1, 22]@21 ] Object evaluation failed: object does not contain group {:choice_combiner=>"|"@58, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"baz"@63}, :primitive_rule=>{:string=>"string"@68}}}, {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]}]} for  rule at '"foo"@21' ( line 1 column 22 ) [ [{:group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"foo"@21}, :primitive_rule=>{:integer_v=>"integer"@26}}}, {:sequence_combiner=>","@34, :member_rule=>[{:not_annotation=>"not"@38}, {:member_name=>{:q_string=>"baz"@44}}, {:primitive_rule=>{:string=>"string"@49}}]}]}, {:choice_combiner=>"|"@58, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"baz"@63}, :primitive_rule=>{:string=>"string"@68}}}, {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]}]}] ] from rule at '"bar"@3' ( line 1 column 4 )
[ 1:[1, 4]@3 ] Object evaluation failed: object does not contain group {:sequence_combiner=>","@14, :group_rule=>[{:group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"foo"@21}, :primitive_rule=>{:integer_v=>"integer"@26}}}, {:sequence_combiner=>","@34, :member_rule=>[{:not_annotation=>"not"@38}, {:member_name=>{:q_string=>"baz"@44}}, {:primitive_rule=>{:string=>"string"@49}}]}]}, {:choice_combiner=>"|"@58, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"baz"@63}, :primitive_rule=>{:string=>"string"@68}}}, {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]}]}]} for  rule at '"bar"@3' ( line 1 column 4 ) [ [{:member_rule=>{:member_name=>{:q_string=>"bar"@3}, :primitive_rule=>{:string=>"string"@8}}}, {:sequence_combiner=>","@14, :group_rule=>[{:group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"foo"@21}, :primitive_rule=>{:integer_v=>"integer"@26}}}, {:sequence_combiner=>","@34, :member_rule=>[{:not_annotation=>"not"@38}, {:member_name=>{:q_string=>"baz"@44}}, {:primitive_rule=>{:string=>"string"@49}}]}]}, {:choice_combiner=>"|"@58, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"baz"@63}, :primitive_rule=>{:string=>"string"@68}}}, {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]}]}]}] ] from rule at '"bar"@3' ( line 1 column 4 )
Failure: object does not contain group {:sequence_combiner=>","@14, :group_rule=>[{:group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"foo"@21}, :primitive_rule=>{:integer_v=>"integer"@26}}}, {:sequence_combiner=>","@34, :member_rule=>[{:not_annotation=>"not"@38}, {:member_name=>{:q_string=>"baz"@44}}, {:primitive_rule=>{:string=>"string"@49}}]}]}, {:choice_combiner=>"|"@58, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"baz"@63}, :primitive_rule=>{:string=>"string"@68}}}, {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]}]}]} for  rule at '"bar"@3' ( line 1 column 4 ) [ [{:member_rule=>{:member_name=>{:q_string=>"bar"@3}, :primitive_rule=>{:string=>"string"@8}}}, {:sequence_combiner=>","@14, :group_rule=>[{:group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"foo"@21}, :primitive_rule=>{:integer_v=>"integer"@26}}}, {:sequence_combiner=>","@34, :member_rule=>[{:not_annotation=>"not"@38}, {:member_name=>{:q_string=>"baz"@44}}, {:primitive_rule=>{:string=>"string"@49}}]}]}, {:choice_combiner=>"|"@58, :group_rule=>[{:member_rule=>{:member_name=>{:q_string=>"baz"@63}, :primitive_rule=>{:string=>"string"@68}}}, {:sequence_combiner=>","@75, :member_rule=>[{:not_annotation=>"not"@79}, {:member_name=>{:q_string=>"foo"@85}}, {:primitive_rule=>{:integer_v=>"integer"@90}}]}]}]}] ] from rule at '"bar"@3' ( line 1 column 4 )

And here is the validation working with that logic:

→  jcr -v -R '{ "bar":string, ( ( "foo":integer , @{not} "baz":string ) | ( "baz":string , @{not} "foo":integer ) ) }' -J '{ "bar":"thing", "foo":2  }'
"Ruleset Parse Tree"
[{:object_rule=>
   [{:member_rule=>
      {:member_name=>{:q_string=>"bar"@3},
       :primitive_rule=>{:string=>"string"@8}}},
    {:sequence_combiner=>","@14,
     :group_rule=>
      [{:group_rule=>
         [{:member_rule=>
            {:member_name=>{:q_string=>"foo"@21},
             :primitive_rule=>{:integer_v=>"integer"@26}}},
          {:sequence_combiner=>","@34,
           :member_rule=>
            [{:not_annotation=>"not"@38},
             {:member_name=>{:q_string=>"baz"@44}},
             {:primitive_rule=>{:string=>"string"@49}}]}]},
       {:choice_combiner=>"|"@58,
        :group_rule=>
         [{:member_rule=>
            {:member_name=>{:q_string=>"baz"@63},
             :primitive_rule=>{:string=>"string"@68}}},
          {:sequence_combiner=>","@75,
           :member_rule=>
            [{:not_annotation=>"not"@79},
             {:member_name=>{:q_string=>"foo"@85}},
             {:primitive_rule=>{:integer_v=>"integer"@90}}]}]}]}]}]
"Ruleset Map"
"Evaluating Root:"
" { \"bar\" : string ,  (  ( \"foo\" : integer ,  @{not} \"baz\" : string ) |  ( \"baz\" : string ,  @{not} \"foo\" : integer ) ) }"
[ 1:[1, 4]@3 ] Evaluating object rule starting at '"bar"@3' ( line 1 column 4 ) against data: {"bar"=>"thing", "foo"=>2}
[ 1:[1, 4]@3 ] object definition:  { "bar" : string ,  (  ( "foo" : integer ,  @{not} "baz" ... data: {"bar"=>"thing", "foo"=>2}
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 4]@3 ] Evaluating member rule for key 'bar' starting at '"bar"@3' ( line 1 column 4 ) against  data: "thing"
[ 2:[1, 4]@3 ] member definition: "bar" : string data: ["bar", "thing"]
[ 3:[1, 9]@8 ] Evaluating value rule starting at '"string"@8' ( line 1 column 9 )
[ 3:[1, 9]@8 ] value definition: string data: "thing"
[ 3:[1, 9]@8 ] Value evaluation is true
[ 2:[1, 4]@3 ] Member evaluation is true
[ 1:[1, 4]@3 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 1:[1, 4]@3 ] rule repetition min = 1 max = 1 repetition step =
[ 2:[1, 22]@21 ] Evaluating object rule starting at '"foo"@21' ( line 1 column 22 ) against data: {"bar"=>"thing", "foo"=>2}
[ 2:[1, 22]@21 ] object definition:  {  ( "foo" : integer ,  @{not} "baz" : string ) |  ( "ba ... data: {"bar"=>"thing", "foo"=>2}
[ 2:[1, 22]@21 ] rule repetition min = 1 max = 1 repetition step =
[ 3:[1, 22]@21 ] Evaluating object rule starting at '"foo"@21' ( line 1 column 22 ) against data: {"bar"=>"thing", "foo"=>2}
[ 3:[1, 22]@21 ] object definition:  { "foo" : integer ,  @{not} "baz" : string } data: {"bar"=>"thing", "foo"=>2}
[ 3:[1, 22]@21 ] rule repetition min = 1 max = 1 repetition step =
[ 4:[1, 22]@21 ] Evaluating member rule for key 'foo' starting at '"foo"@21' ( line 1 column 22 ) against  data: 2
[ 4:[1, 22]@21 ] member definition: "foo" : integer data: ["foo", 2]
[ 5:[1, 27]@26 ] Evaluating value rule starting at '"integer"@26' ( line 1 column 27 )
[ 5:[1, 27]@26 ] value definition: integer data: 2
[ 5:[1, 27]@26 ] Value evaluation is true
[ 4:[1, 22]@21 ] Member evaluation is true
[ 3:[1, 22]@21 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 3:[1, 22]@21 ] rule repetition min = 1 max = 1 repetition step =
[ 3:[1, 22]@21 ] No member 'baz' found in object.
[ 4:[1, 39]@38 ] Evaluating member rule for key '' starting at '"not"@38' ( line 1 column 39 ) against
[ 4:[1, 39]@38 ] member definition:  @{not} "baz" : string data: [nil, nil]
[ 4:[1, 39]@38 ] Not annotation changing result from false to true
[ 4:[1, 39]@38 ] Member evaluation is true
[ 3:[1, 22]@21 ] Found 1 matching members repetitions in object with min 1 and max 1
[ 3:[1, 22]@21 ] Object evaluation is true
[ 2:[1, 22]@21 ] Object evaluation is true
[ 1:[1, 4]@3 ] Object evaluation is true
Success!

[codalogic]

While I agree with the idea that unknown members are ignored, at the moment I don't agree that known members that are superfluous are ignored. I think it comes down to whether | is a choice, in which you can only pick one, of n items, or whether it's an inclusive OR. xs:choice in XML schema would be the former. I think it becomes more of an issue when the data types are objects, such as in:

$message = { 
        "id" : int16, ( "request" : $request | "response" : $response ) }

But I'm happy to leave that aside for the time being and see how things develop.

[anewton1998]

Hmmm... interesting point.

[anewton1998]

I'm trying to tease out of this if this is an issue for arrays, either ordered and unordered, and I don't think so. But that is more a property of the array not allowing extra items than it is being inclusive or exclusive or.

[codalogic]

I suppose you could have an array spec something like:

[ integer *, (ipv4 * | ipv6 *), string * ]

You wouldn't expect to allow both ipv4 and ipv6 addresses then.

I'll confess I haven't looked at your array code yet, but my feeling at the moment is that ordered arrays need to be modelled more like regular expressions (except rather than characters you're matching types). I recall that Kernighan and Pike came up with a simple, cut-down RE engine that may be sufficient for this (I think this is it http://www.cs.princeton.edu/courses/archive/spr09/cos333/beautiful.html).

Unordered arrays are probably more like objects with members that have a null member name.

[anewton1998]

I don't get the reference to the regex engine, but I've got a local branch going to work on XOR. So far I've modified object evaluation without too much trouble (minus new tests to prove it out, but all the old tests pass). I suspect that both ordered and unordered arrays would be the same.

And I think you've hit the arrays from the right angle, and my gut feeling is that there won't be much change between ordered and unordered arrays. Let me play and see what I come up with.

That said, it would be great if the choice combiner signifies XOR in all cases (arrays and groups and objects), not just for objects.

[anewton1998]

After some thought and tinkering, I believe xor for objects and unordered arrays are similar. I've got it implemented for objects, and my guess is that it will take little effort for unordered arrays once I have the time to get to it.

That being said, I'm unsure any special care needs to be taken for ordered (i.e. normal) arrays. Or said another way, there is no difference between inclusive or and exclusive or with arrays. And the same is true of primitives.

$thing =: ("foo" | "bar" )

Since a primitive can be only one value, there is no opportunity to have a comparison of two values. And I think the same is true of arrays, since an ordered array is always comparing an item at a specific position. Take the following JSON array:

[ 1, "foo", "fuzz" ]

The item at position 2 is "foo", and there can only be one item at position 2.

Now here is where things get interesting. Does the following rule match against the array above?

$a1 = [ integer , ("foo" * | "bar" * ), string *]'

What about this rule?

$a2 = [ integer , ("foo" + | "bar" + ), string *]

From what I believe (and what my software is telling me), $a1 fails while $a2 does not. And the reason $a1 fails is because "bar" * is true as ZERO or more "bar" are present at position 2, thus failing the XOR condition since both "foo" * and "bar"* are true.

[codalogic]

The idea that $a1 = [ integer , ("foo" * | "bar" * ), string *] should always fail seems counter intuitive to me, even though I'm not able to disagree with your logic. I guess what they'd really be trying to say is $a1 = [ integer , ("foo" + | "bar" + ) ?, string *]. My worry is that people will be surprised and confused when the former always fails. It would be nice if we could tweak the rules so that it didn't, but I'm not sure how to at the moment.

Maybe we need some sort of tri-state logic involving false, soft-true and hard-true. A thing that is absent, and can be absent is a soft-true, whereas something that is present and is permitted to be present is a hard-true. Using ! to represent only one can be true we'd have a logic table (which could be iteratively applied to expressions like a ! b ! c ! d) like:

false ! hard-true = hard-true
hard-true ! soft-true = hard-true
hard-true ! hard-true = false (and shortcut exit)
soft-true ! soft-true = soft-true
false ! soft-true = soft-true
false ! false = false

A final result of false would mean validation failed, but a result of either soft-true or hard-true would mean validation passed.

Any comp sci whizzes out there that can comment?!

[johnwcowan]

On Thu, Sep 1, 2016 at 8:56 AM, Andrew Newton notifications@github.com
wrote:

Take the following JSON array:

[ 1, "foo", "fuzz" ]

The item at position 2 is "foo", and there can only be one item at
position 2.

Now here is where things get interesting. Does the following rule match
against the array above?

$a1 = [ integer , ("foo" * | "bar" * ), string *]'

What about this rule?

$a2 = [ integer , ("foo" + | "bar" + ), string *]

From what I believe (and what my software is telling me), $a1 fails while
$a2 does not.

I think that's a very strong counterexample to the notion that | (choice)
is XOR rather than IOR. Using the (IMO correct) IOR interpretation, a1
will succeed, because it says "an integer, forllowed by a choice between
zero or more foos and zero or more bars, followed by zero or more strings.
Your example succeeds because it has zero or more foos (in fact, just one),
so the choice is satisfied. Then the final string * picks up the fuzz, and
that's that. Indeed, $a1 is equivalent to [integer, string *].

$a2 succeeds because at least one branch of the choice succeeds: after the
integer, there is one or more foo matching foo+, and then buzz matching
string*.

But both sides can also succeed. If the test string is [13, "buzz"], then
$a1 succeeds because ia choice succeeds if either branch succeeds (IOR).
That is, foo* succeeds because there are no foos, and bar* succeeds because
there are no bars. $a2 however would fail on this test string, because it
requires either a foo or a bar and gets neither.

This is how string regular expressions behave, and we should not violate
the expectation that our regular expressions applied to seuences (arrays or
parts of arrays) will behave exactlly the same way.

[anewton1998]

I agree that regular expressions are more like IOR. See this example:
http://rubular.com/r/6tacOwEuWo

That said, I'm not sure regular expressions are a good ideal as lots of people hate them. Nor do I think following their guidance is particularly helpful here. Let me explain.

If what I mean by ("bar"*|"foo"*) is that I am expecting either ["foo","foo"] or ["bar","bar"] then neither IOR nor XOR deliver:

rule: [ "foo" * | "bar" * ]

example	Expected	IOR	XOR
["foo", "foo" ]	yes	true	false
["foo", "bar" ]	no	true	false
["bar", "bar" ]	yes	true	false

In other words, IOR yields false positives and XOR yields false negatives.

Unfortunately, that leaves us with two options: 1) delve into tri-state logic as @codalogic has hinted at, or 2) simply spell out the pitfall in the spec so people understand the issue.

I'm not a fan of option 1 at present. Therefore I say we go with option 2, and that for the sake of the spec all choice combiners are XOR being that XOR makes sense for objects and unordered arrays and there is no good OR for ordered arrays other than to be consistent.

[anewton1998]

BTW, not that I've thought this through but instead of 3 state logic, perhaps we can add a modifier to the repetition that indicates exclusivity. As in *! or +! which would mean zero or more exclusively or one or more exclusively. Just a thought.

[johnwcowan]

On Fri, Sep 2, 2016 at 11:18 AM, Andrew Newton notifications@github.com
wrote:

If what I mean by ("bar"|"foo") is that I am expecting either
["foo","foo"] or ["bar","bar"] then neither IOR nor XOR deliver:

rule: [ "foo" * | "bar" * ]
example Expected IOR XOR
["foo", "foo" ] yes true true
["foo", "bar" ] no true false
["bar", "bar" ] yes true true

I think we must be talking past each other. The rule reads "any number of
foos, including zero, or else any number of bars, including zero" when I
read it. So ["foo","foo"] satisfies the rule, as does ["bar","bar"], as
does [](because it is IOR, not XOR). But ["foo","bar"] cannot satisfy it,
because once you have a "foo" you are committed to all the rest (if any)
being "foo". This is exactly your "expected" column.

So I do not understand what you mean by IOR and XOR, because it does not
seem to be what I mean. I mean by IOR that either or both sides of a choice
are satisfied, and by XOR that either (but not both) sides of a choice are
satisfied. The precedents are for IOR, and I prefer IOR.

John Cowan http://www.ccil.org/~cowan cowan@ccil.org
And through this revolting graveyard of the universe the muffled,
maddening beating of drums, and thin, monotonous whine of blasphemous
flutes from inconceivable, unlighted chambers beyond Time; the
detestable pounding and piping whereunto dance slowly, awkwardly, and
absurdly the gigantic tenebrous ultimate gods --the blind, voiceless,
mindless gargoyles whose soul is Nyarlathotep. (Lovecraft)

[anewton1998]

Sorry, I was fighting with the markdown syntax for table and made a mistake. I subsequently fixed it but that probably did not come through in email. Here it is with the empty case:

rule: [ "foo" * | "bar" * ]

example	Expected	IOR	XOR
["foo", "foo" ]	yes	true	false
["foo", "bar" ]	no	true	false
["bar", "bar" ]	yes	true	false
[ ]	??	true	false

I see your point, but I need to think about this because it appears that with IOR getting around the mixed case and the empty case take a lot more work than with XOR.

[codalogic]

I knocked up the following Ruby:

def headings()
    print "i(f*|b*)s  i(?:f+|b+)s\n"
end

def evaluate( ip )
    print (/i(f*|b*)s/ =~ ip) ? '  true     ' : '  false    '
    print (/i(f+|b+)s/ =~ ip) ? '  true     ' : '  false    '
    print " : #{ip}\n"
end

headings
evaluate 'is'
evaluate 'iffs'
evaluate 'ibbs'
evaluate 'ifbs'

and got the following results:

i(f*|b*)s  i(?:f+|b+)s
  true       false     : is
  true       true      : iffs
  true       true      : ibbs
  false      false     : ifbs

Significantly (for me anyway :-) ), is yields true for i(f*|b*)s.

(Sorry must dash - people coming!!!)