Add syntax and semantics for metadata

[cpcloud]

Many systems have metadata such foreign keys, primary keys, sortedness, and uniqueness.

kdb+ has an additional attribute called "grouped" which indicates a large amount of repetition.

http://code.kx.com/wiki/JB:QforMortals2/tables#Attributes

Questions:

• What metadata do we want to support?
• What syntax do we want?
• How general do we want attributes to be?
  • Should ndarrays support sorted and unique?
  • I think that only collections should have these attributes
• Should we be able to set these attributes later, i.e., after a computation has occurred, or just when we're constructing the datashape?

I'll be posting what other systems do regarding these, and any interesting tidbits that turn up.

Personally, I like the name "attributes", so that's what I call them when discussing them here.

[mrocklin]

I like the approach of looking at other systems. I think that this would probably be a separate project from datashape (not sure if that was your intention or not), dataattributes?

[cpcloud]

it would likely be a separate project, and would probably depend on datashape

[cpcloud]

Here's an example from kdb+ of how foreign keys are specified and used:

q)kt:([eid:1001 1002 1003] name:`Dent`Beeblebrox`Prefect; iq:98 42 126)
q)kt
eid | name       iq
----| --------------
1001| Dent       98
1002| Beeblebrox 42
1003| Prefect    126
q)tdetails:([] eid:`kt$1003 1001 1002 1001 1002 1001; sc:126 36 92 39 98 42)
q)tdetails
eid  sc
--------
1003 126
1001 36
1002 92
1001 39
1002 98
1001 42
q)select eid.iq, eid.name from tdetails
iq  name
--------------
126 Prefect
98  Dent
42  Beeblebrox
98  Dent
42  Beeblebrox
98  Dent