Building a regional covid atlas

In this tutorial we are going to place ourselves in the mindset of a local health researcher in NYC who is interested in childcare during the pandemic. To do this we will be using data from the NYT to explore Covid Data in NYC and data from the NYC open data portal about local childcare providers.

We will

• Load daily covid data at the county level from our data library
• Create filters to select and aggregate the daily rates over some time period
• Hook those filters up to controls.
• Make maps of the filtered data for cases, deaths and vaccinations
• Lay these out in a meaningful way
• Enable click and hover based tool tips for the data
• Create time series graphs for the selected county
• Bring in local POI data from the ny state open data portal.

Let's get started!

Creating the application.

First thing we need to do is login into the matico dashboard. To do some, go to the Matico app server and click the Login link. You will be asked to sign in with either Github, Google or Facebook social login.

Once you have logged in you will see options to either view, edit or delete your previous apps and to create new ones from some templates.

We are going to build this app from scratch so select the Blank template option and give the app a meaningful name and description

This should land you in the Matico App Builder interface where we will be doing all of our work today.

Loading the data

To be able to do almost anything, we are going to need to load in data. On the left of the screen you will see Icons that allow us to access different actions

• App overview : See and manipulate the overview of the application
• Datasets : See, add, edit and remove Datasets that we will use im the application
• Variables : See what variables are available in the application for interaction
• Errors : Any errors in the application
• Spec : View the generated specification (you wont need this)

Go ahead and click the datasets button

When you do this it will pop out the datasets pane. It will give you the option to Add a Dataset or add a Transform. Go head and select Add Dataset.

We can bring in data from multiple different sources with Matico, from links to files or API's public on the internet in common formats, to data from open data portals and even data you can upload yourself.

We have curated some common datasets that we use into a Data Library. Select that tab to see what is in there.

Let's load in the daily dataset of covid cases, deaths and vaccinations at the county level in New York. Select New York from the drop down list and select New York.

The dataset will appear in the dataset list and its status icon should turn from yellow to green as the dataset loads in successfully.

Clicking on the New York dataset will let us see the columns available in that table.

The format of the table is one row per county per day. The day is represented by the date column and the county by the fips column. Importantly, this table only includes tabular data but no geospatial data so currently we would be unable to put the data on a map.

It does have a fips code however which uniquely identifies the county, so if we have a geospatial dataset that also has a fips code, we can join them to map this data.

We have this dataset in our data library again. Click "Add Dataset" again and select the "Data Library" and instead of "Covid" click census and select "Census Counties (Combined Metro NYC)" to load that dataset in.

Similarly, clicking on this dataset in the dataset view will show the table associated with the dataset. It contains information about each county, the geometry for the county and a "GEOID" column which represents the fips code of the county. This is what we will want to join the data on.

Manipulating the data.

To join the data and perform other actions with it we will use Matico's "Dataset Transform" feature. This lets us easily filter, join, aggregate and transform your datasets right from the browser.

In this case we want to be able to get a table of the total number of cases, deaths and vaccinations over some timespan at the county level.

Click on "Add Transform" in the datasets view. Name the transform something like "cases_over_time" and give it a good description to help you remember what the data represents. Most importantly select "New York" as the base dataset for the transform.

Click on the new transform to bring up the transform editor. Currently, we have no operations on the data and so we just see the same table as the original dataset.

Lets go ahead and add a filter transform. To do so select "Add Transform Step" and then select 'filter'.

This will add a new filter step. We can filter by multiple different columns and factors at the same time. We want to filter by date so to start filtering select the "Date Range" filter.

From the column drop down, select the date column and then play around with the min and max values to filter the data. As you change those values, the number of results in the table should change to reflect the filtering.

This has just filtered the dataset but we still have 1 row per day per county. To get the total number of each statistics over the time period we are going to have to aggregate the data.

So let's add another transform, this time an aggregation using the "Aggregate" transform.

Select the fips value to group by and add three aggregates as follows

This will add up cases, deaths and vaccines in the time span

Finally we need to join the dataset with the geometry dataset to be able to represent it on a map. To do this add one final transform step of type join.

First select the dataset we want to join with, in this case it will be the "Census Counties" dataset. The covid dataset has a FIPS code column which matches up to the GEOID column on the Census Counties dataset. So select the left join column as "fips" and the right join column as "GEOID".

This should join the filtered and aggregate data with the geometry file and give us our final dataset.

Building the interface

Now that we have our datasets worked out we can start with the fun part of the process. Building out our application.

First let's add a new page to the application. To do so, click the plus button on the application navigation bar.

This will create a homepage for us to start adding content to. Click on the home icon and you should see the right hand editor pane change to display the options for the page.

Here we can change the name of the page, it's icon and it's path within the applications url. For now we will leave these options as they are and instead add our first pane. To do so click the add button under the pane submenu.

This will show a list of the available pane types. Lets name the map "covid county map" and add it to the page by selecting the map pane option.

Doing so will have added the interactive map to the page. If we click on the map we will notice that it brings up the editor menu for the map on the right hand side of the screen and selects the map pane in green.

We can reposition the map or resize it by using the controls on the border of the map.

Or we can also set the location and size using the Layour portion of the map pane editor. We will want this map to be full screen so select that preset from the layout section.

The map should be displayed full screen now.

Lets also go ahead and change the style of the map we are using. To do this, locate the layers section of the editor and change the basemap dropdown to "CartoDBDarkMatter". The map should change to a darker themed base map

Finally, lets get some data on our map.

In the layers section, click "Add Map Layer" which will bring up a small menu allowing you to name the new layer and specify what dataset it should use. We can use either raw datasets or dataset transform as sources for our layers. We will want this layer to show the filtered, aggregated and joined data we prepared earlier, so select that transformed dataset name from the drop down and hit "Add to Map"

You should see a red polygon appear on the map. The dataset is for all of New York State so we are a little too zoomed in to see it all. To zoom out, simply click on the map pane and use the zoom controls to zoom out till you can see the full dataset. Drag the map to centre the dataset, then in the "Map Bounds" part of the editor, press the "Set Map Bounds from Current View" button. This will set the map to always start at this zoom level and location.

Great we can see the polygons but we want to colour them by the number of covid cases. To do this we can use Maticos style by data functionality. In the Layers section of the editor, you will see the layer we just added called "covid layer". Click that name to bring up the editor for the layer.

There are lots of different options here but lets focus on the Fill section as this is the one we will use to change the colour of the counties. First lets play with manually changing the fill colour. Press the "select colour" button to bring up the appropriate menu.

Change the sliders to select a color you like and the counties should change to match that color. This is great but it styles all of the counties the same regardless of their covid values. To setup styling by data, we need to select the value we want to use from the drop down menu next to the color selector.

Select total cases from that menu to color each county by it's total number of cases. This will also add a legend to the map to help you interpret the colors.

Tooltips

This is great but it's all a bit static just now. Lets add some interactivity in the form of tooltips. In the tooltip section of the layer editor, click the add tooltip button.

Select "total_cases'' from the "Data Column" drop down and change the Text Label to "Total Cases". Now when we hover over a county, or click on the county we should see a tooltip popup showing the total number of cases in that county. You can add as many different data points to the tooltips as you like. Repeat the steps above to add one for total deaths and total vaccinations.

Interactivity

Currently our map shows the total values for covid cases, deaths and vaccinations over all time. We ideally would want to be able to select a time range over which to see the results. To do this we can add a control to our page.

On the far left hand side of the page, in the menu bar, select the first icon to bring up the application outline view. This shows each page and the panes it contains in a hierarchical view. Currently, we only have the "Home Page" and the "County map displayed".

If we hover over the Home page name, we should see an "add button" pop up. Name the new pane "date slider" and select the "Date / Time Slider" option from the bottom right. Once you do this, you should see the new pane added to the layout view and to the page.

We next need to configure the date time slider to use data from our datasets. Either click on the new time slider pane itself or on the name of the time slider pane on the outline view to open it's editor.

We need to tell the slider which data source and data variable to use to populate the time slider. In the "Data Source" section of the editor, select "New York" as the dataset and the "date" column from the column drop down. This should fully configure the slider and it should fully display on the page.

Use the controls on the border of the slider to resize and move it to roughly this position:

The slider should be maxed out to the full range of our covid dataset. You can select a sub range either by dragging the handles or clicking on the calendar icon and using the picker.

Doing so just now though wont change our data on the map because we need to hook up the slider to our dataset transform.

To do so click on the dataset button on the far left menu and select the "filter_covid_data" transform to bring up the transform editor.

We want to filter the data using the values from the slider. To do this we go back to that first filter step. When we initally set it up, we manually set the start and end date of the filter. Instead we want this information to come from the slider. We can set this up by clicking the button next to the Start date input.

This puts the filter into variable mode, select the date range filter from the "Select variable to use" dropdown and then select the "min" property from the property dropdown.

Repeat this for the "End Date" but instead select the "max" property.

Close the Dataset Transform menu and try changing the date slider to see the values on the map and the tooltips change to reflect the total cases, deaths etc in the range.

Charts and selection

Lets add a new feature to our application. We would love for users to have the ability to click on a county and see the full history of cases in that county.

To do that we can define a new dataset transform to create the data we need. The steps we will need for this transform are

1. Select the correct county from the filtered dataset
2. Join this with the raw data

Let's set that up now. Create a new Dataset Transform as we did before, call it "selection transform" and select the "filtered_covid_data" as the base dataset. We can use either datasets or dataset transforms as the bases of new transforms.

Add a filter step and add a Category filter. Select the filtering "Column" as "_matico_id" and in the "Add entries in the text field below" press the "Fx" button to set up a variable binding and select the covid click layer.

This will filter the dataset based on the county that is clicked on in the map.

Next add a join step and set the "Join to" option to "New York", set the join type to "Left" and set the left join column and right join column to both by "fips". This will join the selected county to its original data.

Once the transform is setup, close the transform editor.

Finally we need to display this data somehow in the app. Lets add a new chart to the app by going back in to the outline view and adding a new Pane called "selection chart" of type "Line Chart".

Click the Line Chart pane to open it's editor. Set the Data Source as the "selection transform" and select the x column to be "date" and the y column to be "cases".

Now when we click on a county, the chart should show that counties covid history.

Bringing in public data

We are almost done but before we wrap up the application, let's bring in some data about the locations of child care facilities in New York state to add context to the map.

Open the dataset menu on the left hand menu bar again and click add new dataset. This time we are going to use the "Socrata Datasets" option to load data in from an open data portal.

From the potal drop down select the New York State portal : "data.ny.gov". This will load a list of available datasets. From the datasets field search for "Child Care" and select the "Child Care Regulated Programs" dataset from the menu.

We need to tell Matico what columns to use for the geometry of the new dataset. To do so, select the "Add Geometry" column and select the Latitude column to be "Latitude" and the Longitude column to be "Longitude". Click Load dataset and the dataset should load in as before.

Click on the map to bring up its editor and add a new layer called faclilitie and select the "Child Care Regulated Programs" as its datasource.

This will add the point layer showing the facilities below the county layer. Clicking on the context menu next to the facilities layer will let us move it above the polygon layer.

We can use the Layer editor to change the color and size of the points to something nicer and add a tooltip for facility name and type.

Adding an info page and publishing the app

That's about it for building out the app itself. Users finding it on the internet might need a bit more context as to what it is showing for it to be useful. One of the features of Matico is allowing us to add multiple pages to an application so lets go ahead and add an info page.

Click on the outline tab and hover over the "Page Outline" text and an "Add Page" button should appear. Clicking it should add a "Page 1" to the overview. Click on "Page 1" to open it's editor.

Search for info in the icon field to change the page icon to info, change the name of the page to Info and change its path to info.

Click the info button on the application nav bar to make sure we are on the correct page and then click "Add Pane". Select "Text Pane" and maximise it to full screen as we did with the map.

Then we can write anything we want in the text area using the rich text controls. Here we add some flavour text and a source list, but you can add whatever you like including links, text, equations and videos.

Once we are all done we can visit the published app by going back to our app list and clicking view. This will take us to a URL showing the application without the editor that can be shared with researchers, the media or the public.

At any time we can edit the app and the changes will be reflected at the live url.

Next steps

• Explore adding other datasets and other pane types
• Try filtering, aggregating and joining the data in different ways
• Add a header text area to the map page to announce the project.
• Play with different layouts for the app. Check out the Main Side Layout to place charts next to the map rather than on top of it.