{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "# Boulder: missing observations stay missing\n\nThe same Aner code reads a local CSV in `.aner`, VS Code `.anernb` and standard Jupyter `.ipynb`. Use the matching development runtime and extension or optional Jupyter kernel. All counts and positions are invented teaching data; these are not monitoring stations or municipal observations.\n\nDownload `plot_boulder_missing.csv` beside this example's folder tree. Run from the project root containing `examples/fixtures/plot_boulder_missing.csv`, since paths use the process or kernel working directory. This first dashboard needs no map pack or connection. Its interactive output embeds the snapshot, so viewing saved output does not reread the CSV.\n",
      "id": "dashboard-introduction"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "// Requires the matching dashboard-enabled native development runtime.\n",
        "// Run from the project root containing examples/fixtures/plot_boulder_missing.csv.\n",
        "// Synthetic teaching counts and illustrative positions, not municipal observations.\n",
        "import aner.data\n",
        "import aner.plot\n"
      ],
      "id": "dashboard-imports"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "// Long format: counts may be missing; site/date/position metadata is required.\n",
        "var schema = data.schema()\n",
        "schema = data.column(schema, \"date\", \"String\", false)\n",
        "schema = data.column(schema, \"site\", \"String\", false)\n",
        "schema = data.column(schema, \"longitude\", \"Float64\", false)\n",
        "schema = data.column(schema, \"latitude\", \"Float64\", false)\n",
        "schema = data.column(schema, \"count\", \"Float64\", true)\n",
        "let source = data.scan_csv(\"examples/fixtures/plot_boulder_missing.csv\", schema, missing: \"NA\")\n",
        "let bounded = data.bounded(source, max_scanned_rows: 11713,\n",
        "    max_read_bytes: 8388608, timeout_ms: 5000)\n",
        "let table = data.collect(bounded, max_rows: 11712,\n",
        "    max_bytes: 4194304, batch_rows: 256)\n"
      ],
      "id": "dashboard-csv-read"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "## Read gaps without inventing zeros\n\nThe CSV has 35 rows for six sites between October 12 and 18, 2026. It deliberately omits every Wednesday row and Creek's Tuesday row, uses `NA` for Campus on Saturday and East on Friday through Sunday, and records a measured **0** for North on Sunday. Only the count column is nullable. Metadata remains required and consistent.\n\nThe table recipe expands the inclusive seven day calendar. Its 42 expected site/day observations contain 31 measured counts and 11 missing values. Wednesday is a gap in every daily trend. Across the full week, every site's **Period total** is **N/A**; the separate **Observed subtotal** is 13,780 with **Coverage** 31/42. The per site observed subtotals are Creek 3,100 (5/7), Pearl 3,250 (6/7), Campus 3,220 (5/7), North 1,350 (6/7), East 1,020 (3/7) and Chautauqua 1,840 (6/7).\n\nTry these selections:\n\n- October 12 only, all sites: period total 2,220 and coverage 6/6.\n- October 14 only, all sites: period total and observed subtotal N/A, coverage 0/6, and no daily marks. Neutral map markers retain each site's position.\n- East, October 17 to 18: both observations are missing, with N/A summaries and coverage 0/2.\n- North, October 18 only: measured period total 0, observed subtotal 0 and coverage 1/1. Its marker has zero area; the site remains available in the table and checklist.\n- Pearl, October 17 to 18: complete weekend total 1,250, but weekday mean and change are N/A because no weekdays are selected.\n\nA line breaks at a missing count. Period totals and group means require complete coverage rather than treating unknown data as zero or silently reporting a partial total. Observed subtotal and coverage remain separate. Unknown map totals use a neutral 5 px marker. Nonmissing totals use fixed proportional area; zero has zero area. Weekday/weekend change additionally needs both complete groups, a positive weekday baseline and a finite ratio.\n\nUnquoted `NA` is missing because the CSV scan says `missing: \"NA\"`. Empty fields are missing only with an explicit `missing: \"\"` scan setting; this example does not change the reader's strict token policy. A missing site/date row also becomes a missing count. Dates absent for all sites are filled only inside the first/last represented dates; a wholly absent site or an unrepresented leading/trailing interval cannot be inferred. Duplicate rows, missing date/site/position metadata, inconsistent positions and invalid nonmissing counts remain errors. The maximum span is 366 calendar days, with up to 32 sites.\n\n## Select from the availability heatmap\n\nThe **Observation coverage** grid shows every site and date in the captured snapshot, even outside the active filter. Color means observed or missing; it does not encode count magnitude. Measured **0** is observed and `NA` is missing. Tooltips and accessible labels report each site's date, count or N/A and observation state. Active cells are highlighted while the full context stays visible. Snapshot coverage stays visible in this grid; selected range coverage remains in the status and summary table.\n\nClick a cell to focus its site and that single day; `Ctrl/Cmd/Shift-click` toggles that site and still focuses the day while keeping other selected sites. Row header buttons focus or modifier toggle a site without changing the date range. Date inputs and the site checklist highlight the corresponding grid cells and update the map, bars, trends and summary together. Arrow keys and Home/End move focus; Enter or Space activates the focused cell. Large calendars scroll within the grid's table.\n\nTry a Wednesday cell: all six sites are missing that day, so the focused site and day have N/A with coverage 0/1 rather than zero. Add sites with modifier click or the checklist to inspect coverage 0/6. Then select North on Sunday: its measured zero has coverage 1/1, while East on that day remains missing. The grid keeps both cells visible so the distinction remains clear.\n\n## Brush a date range and several sites\n\nWith a mouse, drag a rectangle across coverage cells. Release to select every included site row and the inclusive date window together. Reverse dragging gives the same bounds; missing and measured zero cells remain selectable. The map, bars, trends and summary table update only on release. The grid keeps the rest of the snapshot visible.\n\n`Ctrl/Cmd/Shift-drag` adds the rectangle's rows to the sites selected before the gesture and focuses its date window. Modifier click still toggles one site. The preview leaves existing filters unchanged; Escape during the drag or pointer cancellation discards it. Hold near or beyond a table edge to reveal more dates or sites. Releasing outside ends at the nearest visible cell at the current scroll position. Cancellation preserves the filters, while the table may remain scrolled. Touch and pen keep normal table scrolling and single cell taps. The date inputs, checklist, row buttons and keyboard cell controls remain alternatives to dragging.\n\nTry Pearl through North across October 13 to 16: three sites and four dates give N/A period total, observed subtotal 4,480 and coverage 9/12. Then brush North and East on October 18: measured zero and a missing observation give N/A period total, observed subtotal 0 and coverage 1/2. Neither case fills missing values with zero.\n\nNo analysis code changes are needed. Rerun with the updated native runtime and, for `.anernb`, the matching Aner extension; then export a fresh HTML file to bundle brushing. Existing exported pages retain their captured controls. All interactions remain offline with no CSV reread, kernel request or additional pack; the point only example still needs no map pack. The static SVG preserves initial availability without drag controls.\n\nThe same native chart supplies this heatmap in `.aner` HTML export, VS Code `.anernb` and standard Jupyter. The initial static SVG includes availability too. Long calendars continue in stacked sections with repeated headers and section coverage; exact counts remain in cell titles when space is limited. No new analysis call or map pack is needed, and selecting cells never rereads the CSV or calls the kernel.\n",
      "id": "dashboard-reading-guide"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "// No map pack is needed. Positions and daily counts come from the collected table.\n",
        "let dashboard = plot.site_dashboard_table(table)\n",
        "    .title(\"Boulder: missing observations stay missing\")\n",
        "    .caption(\"Synthetic plot_boulder_missing.csv teaching counts, Oct 12-18 2026. NA and omitted records stay unknown; measured zero is retained. Illustrative sites.\")\n",
        "    .size(width: 1000, height: 600)\n",
        "dashboard.show()\n"
      ],
      "id": "dashboard-display"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "## Export exactly the same snapshot\n\nThe three code cells above are byte identical in `.anernb` and `.ipynb` and form the analysis prefix of `plot_site_dashboard_missing.aner`. Run `aner run examples/plot_site_dashboard_missing.aner` from the project root to create `boulder-dashboard-missing.html`. In either notebook, export on demand in a new cell with a fresh filename:\n\n```aner\ndashboard.save(\"boulder-dashboard-missing.html\")\n```\n\nHTML contains the missing values, geometry and controls; it needs no CSV, kernel, server, CDN or packs to view. Saved outputs and HTML use the same gap, N/A, subtotal and coverage rules. `dashboard.html()` returns that standalone document. `dashboard.save(\"boulder-dashboard-missing.svg\")` saves the initial static composite, including its missing data summary. Existing files are never replaced. CSV edits affect only a new read and chart construction; an existing chart keeps its snapshot.\n\nVS Code saved outputs reopen with the matching renderer. Standard Jupyter supplies a sandboxed HTML iframe under its normal notebook trust and script policy; use its SVG fallback or exported HTML if the frontend disables scripts. Filters run locally without a kernel request. Temporary filter or viewport choices are not written back to the native chart or export.\n",
      "id": "dashboard-export-guide"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "## Optional Colorado boundary context\n\nInstall only `usa.co` before executing this optional variant. Keep it outside Run All so the default remains independent of packs:\n\n```aner\nlet context = plot.map(\"usa.co.places\", keys: List(), values: List())\n    .extent(west: -105.34, south: 39.96, east: -105.20, north: 40.09)\n    .projection(\"lambert_azimuthal_equal_area\")\nlet withBoundaries = plot.site_dashboard_table(table, map: context)\n    .title(\"Boulder: missing observations with municipal context\")\n    .caption(\"Synthetic plot_boulder_missing.csv counts; missing observations stay unknown. Generalized Census 2025 municipal boundaries, no streets.\")\nwithBoundaries.show()\nwithBoundaries.save(\"boulder-dashboard-missing-boundaries.html\")\n```\n\nOnly constructing this variant reads the pack. The saved HTML and scene embed its geometry and authored source caption, and work offline without the CSV or pack. Missingness is a data property and is identical with or without boundaries. This remains a daily site dashboard recipe rather than an arbitrary layout or server callback system.\n",
      "id": "dashboard-optional-boundaries"
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "aner",
      "display_name": "Aner",
      "language": "aner"
    },
    "language_info": {
      "name": "aner",
      "file_extension": ".aner",
      "mimetype": "text/x-aner"
    },
    "aner": {
      "status": "example",
      "execution_mode": "persistent-session"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}