Non-natural-number indicators

[ramanshah]

@andburch:

I also noticed that the INDICATOR TOML only has a description, and NA level, and the labels. I assume this means that it expects indicator data to come in integer values?

I think that's correct. The current prototype code seems to embed an assumption that the indicator is natively 1, 2, ..., N, or blank in the input CSV survey dump. That's a bad assumption.

tabulate_univariate_segmentation <- function(data, spec, seg, ind) {
  seg_spec <- spec[["segments"]][[seg]]
  ind_spec <- spec[["indicators"]][[ind]]

  ind_prep <- paste0(ind, "_prep")
  seg_prep <- paste0(seg, "_prep")

  summary_nonblank <- data[!is.na(get(ind_prep)),
                           .(avg_nonblank=mean(get(ind_prep)), num_nonblank=.N), # <-- Depends on ind_prep being NA, 1, 2, ..., N
                           by=.(x=get(seg_prep))]
  summary_blank <- data[is.na(get(ind_prep)), .(num_blank=.N), by=.(x=get(seg_prep))]

  result <- data.table(x=unique(c(seg_spec$output_levels,
                                  seg_spec$na_level)))
  result[summary_nonblank, avg_nonblank := avg_nonblank, on=.(x)]
  result[summary_nonblank, num_nonblank := num_nonblank, on=.(x)]
  result[summary_blank, num_blank := num_blank, on=.(x)]
  result[is.na(num_nonblank), num_nonblank := 0]
  result[is.na(num_blank), num_blank := 0]

  result
}

Our chosen open-data testbed (the StackOverflow survey) breaks this. It does the (very common) thing of having plain English strings for responses instead of numerical Likert levels. We need to enhance the TOML spec and downstream munging/tabulating logic to shim out the responses and translate them from stuff like "1 - Strongly Disagree" to something like a numeric 1 which is amenable to downstream math.

[andburch]

Can we just treat indicators and segments as exactly the same during the prep phases? As far as I can see, we can just repurpose the "munging" phase of prepping segments and apply it directly to indicators, right?

[ramanshah]

I think so. Lemme think it over tho. The basic difference between indicators and segments is that indicators have this kind of Likert topology (where Strongly Disagree is farther from Disagree than Neutral) while segments do not have this kind of ordering. But at the front-end munging, I think that may not be relevant.

[ramanshah]

The more critical thing is to spec out "which Likert-valued 'indicator' columns do we analyze with respect to which polytomous 'segment' columns (in singletons and pairs)?"

I was already worrying about a shortcoming where users would quickly want to blur the lines between indicators and segments, because it's a judgment call what's independent and what's dependent in any kind of survey analysis. That already got me wondering about how to treat indicators as segments in meaningful analyses. For example, it's natural to look for correlations between different satisfaction measures in a population.

I am thinking whatever change we make here ought to support that need too.

I wonder if you have clear ideas how to do this? The hazy idea I have is something like a spec where the big part is "here are the columns and how to reduce them to 1, 2, ..., N, NA" and then a smaller part is "here are the columns to regard as Likert-valued dependent values, and here are the columns to regard as polytomous-valued independent variables." Then you'd customize the analysis through some combination of tailoring a spec file or making a possibly unwieldy function call.

We need to make sure that the null response handling is seamless between the two cases. For a segment, NA often gets fused into a "Don't Know" response for the purposes of segmentation. But for an indicator, NA is a special value that comprises the gray wedges in the bubbles. These are not the same treatment.

Don't want to get too far over our skis here, though - perhaps better to just make the smallest possible change to admit the StackOverflow survey and then make bigger architectural changes downstream.