Merge branch 'main' into upsert-nowcast

Author: chinandrew

docs/api/covidcast-signals/fb-survey.md

@@ -325,7 +325,12 @@ also available. These have names beginning `smoothed_w`, such as
 | Signal | Description | Survey Item |
 | --- | --- | --- |
 | `smoothed_accept_covid_vaccine` | Estimated percentage of respondents who would definitely or probably choose to get vaccinated, if a COVID-19 vaccine were offered to them today. **Note:** Until January 6, 2021, all respondents answered this question; beginning on that date, only respondents who said they have not received a COVID vaccine are asked this question. <br/> **First Available:** 2021-01-01 | V3 |
-| `smoothed_covid_vaccinated` | Estimated percentage of respondents who have already received a vaccine for COVID-19. **Note:** The Centers for Disease Control compiles data on vaccine administration across the United States. This signal may differ from CDC data because of survey biases and should not be treated as authoritative. However, the survey signal is not subject to the lags and reporting problems in official vaccination data. <br/> **First Available:** 2021-06-01 | V1 |
+| `smoothed_covid_vaccinated` | Estimated percentage of respondents who have already received a vaccine for COVID-19. **Note:** The Centers for Disease Control compiles data on vaccine administration across the United States. This signal may differ from CDC data because of survey biases and should not be treated as authoritative. However, the survey signal is not subject to the lags and reporting problems in official vaccination data. <br/> **First Available:** 2021-01-06 | V1 |
+| `smoothed_vaccine_likely_friends` | Estimated percentage of respondents who would be more likely to get a COVID-19 vaccine if it were recommended to them by friends and family, among respondents who have not yet been vaccinated. <br/> **First Available:** 2021-01-20 | V4 |
+| `smoothed_vaccine_likely_local_health` | Estimated percentage of respondents who would be more likely to get a COVID-19 vaccine if it were recommended to them by local health workers, among respondents who have not yet been vaccinated. <br/> **First Available:** 2021-01-20 | V4 |
+| `smoothed_vaccine_likely_who` | Estimated percentage of respondents who would be more likely to get a COVID-19 vaccine if it were recommended to them by the World Health Organization, among respondents who have not yet been vaccinated. <br/> **First Available:** 2021-01-20 | V4 |
+| `smoothed_vaccine_likely_govt_health` | Estimated percentage of respondents who would be more likely to get a COVID-19 vaccine if it were recommended to them by government health officials, among respondents who have not yet been vaccinated. <br/> **First Available:** 2021-01-20 | V4 |
+| `smoothed_vaccine_likely_politicians` | Estimated percentage of respondents who would be more likely to get a COVID-19 vaccine if it were recommended to them by politicians, among respondents who have not yet been vaccinated. <br/> **First Available:** 2021-01-20 | V4 |
 
 These indicators are based on questions added in Wave 6 of the survey,
 introduced on December 19, 2020; however, Delphi only enabled item V1 beginning

src/acquisition/covid_hosp/common/database.py

@@ -17,7 +17,7 @@ def __init__(self,
                connection,
                table_name=None,
                columns_and_types=None,
-               additional_fields=tuple()):
+               additional_fields=None):
     """Create a new Database object.
 
     Parameters
@@ -26,20 +26,20 @@ def __init__(self,
       An open connection to a database.
     table_name : str
       The name of the table which holds the dataset.
-    columns_and_types : list[tuple[str, Callable[str, ...]]]
-      List of CSV columns in order of appearance in the database. The first
-      element of each tuple is the CSV column name, and the second element is a
-      function which converts a string into the appropriate datatype for the
-      column.
+    columns_and_types : tuple[str, str, Callable]
+      List of 3-tuples of (CSV header name, SQL column name, data type) for
+      all the columns in the CSV file.
     additional_fields : tuple[str]
-      Tuple of additional fields to include at the end of the row which are not
-      present in the CSV data.
+      List of 2-tuples of (value, SQL column name) fordditional fields to include
+      at the end of the row which are not present in the CSV data.
     """
 
     self.connection = connection
     self.table_name = table_name
+    self.publication_col_name = "issue" if table_name == 'covid_hosp_state_timeseries' else \
+      'publication_date'
     self.columns_and_types = columns_and_types
-    self.additional_fields = additional_fields
+    self.additional_fields = additional_fields if additional_fields is not None else []
 
   @classmethod
   @contextmanager
@@ -152,16 +152,19 @@ def insert_dataset(self, publication_date, dataframe):
 
     num_columns = 2 + len(self.columns_and_types) + len(self.additional_fields)
     value_placeholders = ', '.join(['%s'] * num_columns)
-    sql = f'INSERT INTO `{self.table_name}` VALUES ({value_placeholders})'
-
+    columns = ', '.join(f'`{i[1]}`' for i in self.columns_and_types + self.additional_fields)
+    sql = f'INSERT INTO `{self.table_name}` (`id`, `{self.publication_col_name}`, {columns}) ' \
+          f'VALUES ({value_placeholders})'
     id_and_publication_date = (0, publication_date)
     with self.new_cursor() as cursor:
       for _, row in dataframe.iterrows():
         values = []
-        for name, dtype in self.columns_and_types:
+        for name, _, dtype in self.columns_and_types:
           if isinstance(row[name], float) and math.isnan(row[name]):
             values.append(None)
           else:
             values.append(dtype(row[name]))
         cursor.execute(sql,
-                       id_and_publication_date + tuple(values) + self.additional_fields)
+                       id_and_publication_date +
+                       tuple(values) +
+                       tuple(i[0] for i in self.additional_fields))