CSC8636 – Summative Resit Assessment

Visual Analysis of the Ocean Microbiome

Background

Data visualization has become an important tool for explorative data analysis as well as for presentation and communication of data in many application domains. A domain that has become increasingly data driven over the last decades are biosciences, and in particular when it comes to studies of the microbiome and other genome sequenced data. In this summative assessment, you are asked to design and implement an interactive multiple coordinated views visualization that support analysis of data from a study of the ocean microbiome, using different visualization methods.

The focus of the tasks in the assessment is on visualization of heterogeneous and multivariate (high dimensional) data, interactive visualization and multiple views, and the application of theory and principles in visualization design.

Data context

The oceans are the largest cohesive eco-system on earth, and a greater understanding of this eco-system is important for the preservation of the planet as well as for understanding of how organisms have evolved since life began. The data that you will work with originates from a two-and-a-half-year expedition with the schooner Tara, during which oceanic samples were collected from 210 stations across the world oceans. If you are interested, you can read more about the expedition and ocean microbiome here: https://www.embl.org/topics/tara/

User context

The end user of the visualization that you will develop would typically be a microbiologist or another domain expert in a bioscience field. The aim of their analysis would be to increase their knowledge of the ocean microbiome, and analysis questions of particular interest may for example include:

· Which microbes are detected at the highest levels overall in the oceans?

· Which microbes are detected at the highest levels in certain regions of the oceans?

· Are there differences in microbe detection levels that can be linked to other features of the oceanic samples, for example the geographic region, sample depth etc?

· Are there differences between taxonomic levels, which can be linked to other features of the oceanic samples?

The data

The data that you will work with relates to 135 samples that were taken from different oceanic regions. In the full dataset, the detection levels of 35,650 Operational Taxonomic Units (OTUs) were recorded for the individual samples. Detection levels are sometimes referred to as the abundance of the OTU. OTUs are close approximations of microbial species, which are extracted through clustering of DNA sequences, so you can think of an OTU as being the same as a microbial species (such as a bacterium). The OTUs also have an associated hierarchical taxonomy through the biological classification system (https://en.wikipedia.org/wiki/Taxonomy_(biology) ), and are often converted into higher levels in the taxonomy for analysis, since an OTU name generally has no biological meaning.  In addition to the OTU detection levels, there are a range of contextual data associated with the samples (i.e. metadata).

You have been provided with a cleaned, wrangled, and formatted version of the Tara Ocean data to work with, which should be reasonably straightforward to get started with. This dataset includes a subset of the full OTU dataset, which makes up 80% of the total cumulative abundance of microbes. The subset of OTUs have been merged to the Class level of the biological taxonomy, resulting in a microbial diversity representation consisting of 31 unique Classes. This data has then been merged with contextual metadata about the samples.

In the dataset that you have been provided with (figure 1), each row corresponds to one sample. The first 10 columns correspond to contextual metadata [SampleID, Year, Month, Latitude, Longitude, SamplingDepth, LayerOfOrigin, MarinePelagicBiome, OceanAndSeaRegion, MarinePelagicProvince], and the remaining 31 columns [AEGEAN-245, etc.] are the microbes, merged to Class level. The cells under the microbes represent the relative detection level (relative abundance) of the microbes in samples.

Figure 1: The wrangled TaraOcean dataset to use for the assignment. The first column is the unique ID of each sample, the following nine columns are contextual meta-data for the samples, and the remaining 31 columns contains the detection levels for microbes merged to the Class level of the biological taxonomy.

The assignment

Interactive visualization using multiple coordinated views (90%)

The task of the coursework is to design and implement an interactive multiple coordinated views visualization (dashboard) that support exploration of the Tara Ocean data. The final multiple coordinated views visualization should be designed as a single html file that contains all views, fit on a screen with 1680x1050 resolution, and  should be saved and submitted as one stand alone html page.

The aims of the visualization are to:

1. Help the user understand overall abundance patterns and diversity in the oceans: the user would typically be interested in knowing which the most abundant microbes are, and if there are large variations in abundance between different microbes.

2. Help the user understand some of the abundance patterns and diversity in the oceans in context of the sample meta. data: e.g. Are there differences in abundance profiles between different sample classes?

3. Help the user get an overview of the microbiome while also being able to investigate details and patterns of potential interest in more detail.

For full marks you are expected to include at least three views in your visualization, which are interactively coordinated and display different aspects of the data. You are expected to take accessibility and user diversity into consideration.

Fill in the relevant parts of the submission table to demonstrate your approach to meeting the aims. You need to demonstrate in the table your use of visualization theory and principles in the design, and to justify design choices made. You are expected to also reflect on alternative visualization approaches and methods, and how these could have been used.

Use of language/tools

You must use Python and are recommended to use the Altair and Pandas packages for creating the interactive visualization. You are allowed to use other Python visualization packages, but you must make sure you are able to generate an html version of the multiple coordinated views visualization than can be displayed as a stand-alone web-page without requiring any installations on the user side.

What to submit

· Submit in Canvas a single zip file including:

o Report: A document including the submission table with details and justification of your visualization and design choices, and a list of references to sources used to carry out the project, in pdf format. References in the report must be consistently cited in a standard way.

o Visualization: The html page with your multiple coordinated views visualization (note: this should not be an html version of a Jupyter Notebook, but an html-file saved using Altair’s ‘save’ functionality or similar).

o Code: Your Python code.

Marking Scheme

The following marking scheme will be used