Description:

This file describes the different steps to the quality control based on:

  • the number of UMI (transcripts) detected per cell
  • the number of genes detected per cell
  • the proportion of transcripts related to the genes encoded in the mitochondria, per cell
  • the proportion of transcripts related to the genes encoding ribosomal units, per cell
  • the proportion of transcripts related to stress signature, per cell

Input data: Seurat object containing all cells (filtered count matrix)

Output data: Seurat object annotated for the cells to filter out for downstream analysis, called sobj_TD3A_qc_annotated.rds

Next steps: Cells annotation for cell type, doublet status and cell cycle status

Context

In this file, we used a dataset from the Paiva et al. publication.

The study concerns thymus autonomy:

  • The thymus is an “organ of passage”, critical in its function to the adaptive immune system for the maturation of T cell lymphocytes.

  • This maturation involves two main steps, performed thanks to macrophages:

    • Positive selection : keeping cells that successfully develop react appropriately with MHC immune receptors of the body
    • Negative selection : keeping cells that do not react against natural proteins of the body.
  • Thymus autonomy is a natural mechanism that allows to create T cells in the thymus by differentiation and cell competition, even when normal progenitors from the bone marrow are lacking, in critical conditions.

  • This mechanism is known in its effects, but the cells involved in are not.

  • This study is of importance in the health field, as this mechanism relies on a temporary loss of control of the cell normal functions.

  • The consequence is that if thymus is in autonomy for too long (few weeks), this is a prelude for leukemia !

  • Organism is : mus musculus

  • Individuals are : mice in development, grafted

  • The design corresponds to two conditions (Test / control)

    • Control : thymus from wild type newborn mouse transplanted into wild type juvenile mouse. In this control case, donor T-cells progenitors (DN3) were replaced by host cells 3 weeks after transplantation.
    • Test : thymus from wild type newborn mouse transplanted in KO Rag-/- type juvenile mouse (the KO partially impairs their ability to produce T-cell progenitors in normal amounts). In this test case, donor T-cells progenitors (DN3) were replaced by host cells 9 weeks after transplantation, showing that the donor DN3 cells outlived their normal lifespan by ~6 weeks.

You will mainly work on the KO sample (‘TD3A’). The input data consists in a count matrix, as a gzipped tabular text file, that contains everything needed to create a basic Seurat object : * The expressions counts * The feature names (here, gene symbols) * The barcode names

This matrix has already been filtered for empty droplets.

Pre-requisites

To obtain this file and starts the TD, copy-paste the folder in your project, either using the graphical interface, or in the Terminal:

cp -r /shared/projects/2422_ebaii_n1/atelier_scrnaseq/RMD/Preproc.2 /shared/projects/[your_projects]/TD_Preproc.2

cd /shared/projects/[your_projects]/TD_Preproc.2

The tree should looks like:

ls
## data
## paiva_wt_ko.png
## resources
## TD_QC_metrics.Rmd
## thymus_autonomy.png

Environment

We load the package of interest:

library(dplyr)
library(ggplot2)

.libPaths()
## [1] "/shared/home/aonfroy/R/x86_64-conda-linux-gnu-library/4.4"     
## [2] "/shared/ifbstor1/software/miniconda/envs/r-4.4.1/lib/R/library"

Data

We load the count matrix:

# Set input data
input_data = "./data/GSM4861194_gex_2_raw_gene_expression.tsv.gz"

# Load data
mat = read.table(input_data,
                 header = TRUE,
                 sep = "\t")
mat = as.matrix(mat)
mat = Matrix::Matrix(mat,
                     sparse = TRUE)

dim(mat)
## [1] 31053  4587
mat[c(1:5), c(1:5)]
## 5 x 5 sparse Matrix of class "dgCMatrix"
##         AAACCTGAGACGCTTT.1 AAACCTGAGGCATTGG.1 AAACCTGGTCAACATC.1
## Xkr4                     .                  .                  .
## Gm1992                   .                  .                  .
## Gm37381                  .                  .                  .
## Rp1                      .                  .                  .
## Sox17                    .                  .                  .
##         AAACCTGTCGAGGTAG.1 AAACCTGTCGATCCCT.1
## Xkr4                     .                  .
## Gm1992                   .                  .
## Gm37381                  .                  .
## Rp1                      .                  .
## Sox17                    .                  .

We build a Seurat object from the count matrix:

sobj = Seurat::CreateSeuratObject(counts = mat,
                                  assay = "RNA",
                                  project = "TD3A")
sobj
## An object of class Seurat 
## 31053 features across 4587 samples within 1 assay 
## Active assay: RNA (31053 features, 0 variable features)
##  1 layer present: counts

We do not need the count matrix:

rm(mat)

Global settings

We define the output directory to save the Seurat object in this end of this file:

outdir = "./data"

if (!dir.exists(outdir)) {
  dir.create(outdir, recursive = FALSE)
}

We set the filtering thresholds based on quality control-related metrics. Adjust them as necessary based on the figures.

cut_log10_nCount_RNA = 3
cut_nFeature_RNA = 750
cut_percent.mt = 5
cut_percent.rb = 30
cut_percent.st = 6

We define a nice palette to visualize the QC metrics:

color_palette = c("lightgray", "#FDBB84", "#EF6548", "#7F0000", "black")

For the QC metrics related to the proportion of UMI belongs to a specific gene sets, we need to load the gene sets.

mito_symbols = readRDS("./resources/mus_musculus_mito_symbols_20191015.rds")
mito_symbols
##  [1] "mt-Atp6" "mt-Atp8" "mt-Co1"  "mt-Co2"  "mt-Co3"  "mt-Cytb" "mt-Nd1" 
##  [8] "mt-Nd2"  "mt-Nd3"  "mt-Nd4"  "mt-Nd4l" "mt-Nd5"  "mt-Nd6"  "mt-Rnr1"
## [15] "mt-Rnr2" "mt-Ta"   "mt-Tc"   "mt-Td"   "mt-Te"   "mt-Tf"   "mt-Tg"  
## [22] "mt-Th"   "mt-Ti"   "mt-Tk"   "mt-Tl1"  "mt-Tl2"  "mt-Tm"   "mt-Tn"  
## [29] "mt-Tp"   "mt-Tq"   "mt-Tr"   "mt-Ts1"  "mt-Ts2"  "mt-Tt"   "mt-Tv"  
## [36] "mt-Tw"   "mt-Ty"
ribo_symbols = readRDS("./resources/mus_musculus_cribo_symbols_20191015.rds")
ribo_symbols
##  [1] "Rpsa"   "Rps2"   "Rps3"   "Rps3a"  "Rps4x"  "Rps5"   "Rps6"   "Rps7"  
##  [9] "Rps8"   "Rps9"   "Rps10"  "Rps11"  "Rps12"  "Rps13"  "Rps14"  "Rps15" 
## [17] "Rps15a" "Rps16"  "Rps17"  "Rps18"  "Rps19"  "Rps20"  "Rps21"  "Rps23" 
## [25] "Rps24"  "Rps25"  "Rps26"  "Rps27"  "Rps27a" "Rps28"  "Rps29"  "Rps30" 
## [33] "Rpl3"   "Rpl4"   "Rpl5"   "Rpl6"   "Rpl7"   "Rpl7a"  "Rpl8"   "Rpl9"  
## [41] "Rpl10"  "Rpl10a" "Rpl11"  "Rpl12"  "Rpl13"  "Rpl13a" "Rpl14"  "Rpl15" 
## [49] "Rpl17"  "Rpl18"  "Rpl18a" "Rpl19"  "Rpl21"  "Rpl22"  "Rpl23"  "Rpl23a"
## [57] "Rpl24"  "Rpl26"  "Rpl27"  "Rpl27a" "Rpl28"  "Rpl29"  "Rpl30"  "Rpl31" 
## [65] "Rpl32"  "Rpl34"  "Rpl35"  "Rpl35a" "Rpl36"  "Rpl44"  "Rpl37"  "Rpl37a"
## [73] "Rpl38"  "Rpl39"  "Rpl40"  "Rpl41"  "Rplp0"  "Rplp1"  "Rplp2"
stress_symbols = readRDS("./resources/mus_musculus_stress_symbols_20200224.rds")
stress_symbols
##  [1] "Atf3"     "Cmss1"    "Diaph1"   "Egr1"     "Fos"      "Gls"     
##  [7] "Gm48099"  "lsp90aa1" "lsp90ab1" "lfrd1"    "Jak1"     "Junb"    
## [13] "Kpna1"    "Nup210l"  "Peak1"    "Stat3"    "Actb"     "Camk1d"  
## [19] "Chka"     "Clic4"    "Fodl2"    "Hspa8"    "Jun"      "Jund"    
## [25] "Klf6"     "Litaf"    "Pecam1"   "Ptma"     "Sik3"     "Spag9"   
## [31] "Arih1"    "Azin1"    "Brd2"     "Chd4"     "Ctnnb1"   "Dennd4a" 
## [37] "Elf1"     "G3bp1"    "Ints6"    "Kdm6b"    "Lsmem1"   "Man1a"   
## [43] "Med13"    "Nfkbia"   "Nfkbiz"   "Nop58"    "Piezo1"   "Ppp1r15a"
## [49] "Prkcg"    "Sqstm1"   "Taf4b"    "Tmsb4x"   "Ubc"      "Zfp36"   
## [55] "Abtb2"    "Adamts1"  "Adamts9"  "Ahnak"    "Ankrd28"  "Atp2a2"  
## [61] "Baz1a"    "Btg2"     "Ccdc138"  "Cd44"     "Cdkn1a"   "Elf2"    
## [67] "Ep400"    "Epas1"    "Erf"      "Ern1"     "Fabp4"    "Fosb"    
## [73] "Gm10073"  "Gsk3a"    "H3f3a"    "H3f3b"    "Hivep2"   "Klf4"    
## [79] "Lmna"     "lapkapk2" "Mapre1"   "Msn"      "Mylip"    "Nabp1"   
## [85] "Ncl"      "Nsd3"     "Nufip2"   "Plekhg2"  "Ppp1cb"   "Rnf19b"  
## [91] "Rps20"    "Rtn4"     "Runx1"    "Sertad2"  "Sptan1"   "Top1"    
## [97] "Vcl"      "Zbtb11"

We keep only the gene symbols available in our data:

mito_symbols = intersect(mito_symbols, rownames(sobj))
mito_symbols
##  [1] "mt-Atp6" "mt-Atp8" "mt-Co1"  "mt-Co2"  "mt-Co3"  "mt-Cytb" "mt-Nd1" 
##  [8] "mt-Nd2"  "mt-Nd3"  "mt-Nd4"  "mt-Nd4l" "mt-Nd5"  "mt-Nd6"
ribo_symbols = intersect(ribo_symbols, rownames(sobj))
ribo_symbols
##  [1] "Rpsa"   "Rps2"   "Rps3"   "Rps4x"  "Rps5"   "Rps6"   "Rps7"   "Rps8"  
##  [9] "Rps9"   "Rps10"  "Rps11"  "Rps12"  "Rps13"  "Rps14"  "Rps15"  "Rps15a"
## [17] "Rps16"  "Rps17"  "Rps18"  "Rps19"  "Rps20"  "Rps21"  "Rps23"  "Rps24" 
## [25] "Rps25"  "Rps26"  "Rps27"  "Rps27a" "Rps28"  "Rps29"  "Rpl3"   "Rpl4"  
## [33] "Rpl5"   "Rpl6"   "Rpl7"   "Rpl7a"  "Rpl8"   "Rpl9"   "Rpl10"  "Rpl10a"
## [41] "Rpl11"  "Rpl12"  "Rpl13"  "Rpl13a" "Rpl14"  "Rpl15"  "Rpl17"  "Rpl18" 
## [49] "Rpl18a" "Rpl19"  "Rpl21"  "Rpl22"  "Rpl23"  "Rpl23a" "Rpl24"  "Rpl26" 
## [57] "Rpl27"  "Rpl27a" "Rpl28"  "Rpl29"  "Rpl30"  "Rpl31"  "Rpl32"  "Rpl34" 
## [65] "Rpl35"  "Rpl35a" "Rpl36"  "Rpl37"  "Rpl37a" "Rpl38"  "Rpl39"  "Rpl41" 
## [73] "Rplp0"  "Rplp1"  "Rplp2"
stress_symbols = intersect(stress_symbols, rownames(sobj))
stress_symbols
##  [1] "Atf3"     "Cmss1"    "Diaph1"   "Egr1"     "Fos"      "Gls"     
##  [7] "Gm48099"  "Jak1"     "Junb"     "Kpna1"    "Nup210l"  "Peak1"   
## [13] "Stat3"    "Actb"     "Camk1d"   "Chka"     "Clic4"    "Hspa8"   
## [19] "Jun"      "Jund"     "Klf6"     "Litaf"    "Pecam1"   "Ptma"    
## [25] "Sik3"     "Spag9"    "Arih1"    "Azin1"    "Brd2"     "Chd4"    
## [31] "Ctnnb1"   "Dennd4a"  "Elf1"     "G3bp1"    "Ints6"    "Kdm6b"   
## [37] "Lsmem1"   "Man1a"    "Med13"    "Nfkbia"   "Nfkbiz"   "Nop58"   
## [43] "Piezo1"   "Ppp1r15a" "Prkcg"    "Sqstm1"   "Taf4b"    "Tmsb4x"  
## [49] "Ubc"      "Zfp36"    "Abtb2"    "Adamts1"  "Adamts9"  "Ahnak"   
## [55] "Ankrd28"  "Atp2a2"   "Baz1a"    "Btg2"     "Ccdc138"  "Cd44"    
## [61] "Cdkn1a"   "Elf2"     "Ep400"    "Epas1"    "Erf"      "Ern1"    
## [67] "Fabp4"    "Fosb"     "Gm10073"  "Gsk3a"    "H3f3a"    "H3f3b"   
## [73] "Hivep2"   "Klf4"     "Lmna"     "Mapre1"   "Msn"      "Mylip"   
## [79] "Nabp1"    "Ncl"      "Nsd3"     "Nufip2"   "Plekhg2"  "Ppp1cb"  
## [85] "Rnf19b"   "Rps20"    "Rtn4"     "Runx1"    "Sertad2"  "Sptan1"  
## [91] "Top1"     "Vcl"      "Zbtb11"

Note: A more robust analysis may use the gene identifiers instead of gene symbols.

Fast-processing

To visualize the low quality cells, we generate a projection. Understanding the commands below is the purpose of next course sessions. So just run:

sobj = Seurat::NormalizeData(sobj)
sobj = Seurat::ScaleData(sobj)
sobj = Seurat::FindVariableFeatures(sobj)
sobj = Seurat::RunPCA(sobj)
sobj = Seurat::RunUMAP(sobj, dims = c(1:20))

sobj
## An object of class Seurat 
## 31053 features across 4587 samples within 1 assay 
## Active assay: RNA (31053 features, 2000 variable features)
##  3 layers present: counts, data, scale.data
##  2 dimensional reductions calculated: pca, umap

We can now visualize the cells on a 2D projection:

Seurat::DimPlot(sobj,
                reduction = "umap",
                cols = "black") +
  ggplot2::theme(aspect.ratio = 1,
                 legend.position = "none")

Quality control

Compute metrics

What is already available in the Seurat object ?

head(sobj@meta.data)
##                    orig.ident nCount_RNA nFeature_RNA
## AAACCTGAGACGCTTT.1       TD3A       2813         1594
## AAACCTGAGGCATTGG.1       TD3A       2072         1365
## AAACCTGGTCAACATC.1       TD3A       2025         1341
## AAACCTGTCGAGGTAG.1       TD3A       1877         1241
## AAACCTGTCGATCCCT.1       TD3A       2216         1441
## AAACGGGCACTTACGA.1       TD3A       2445         1428

How do the two first QC metrics vary ?

summary(sobj@meta.data)
##  orig.ident    nCount_RNA     nFeature_RNA 
##  TD3A:4587   Min.   :  502   Min.   :  22  
##              1st Qu.: 1986   1st Qu.:1291  
##              Median : 2397   Median :1476  
##              Mean   : 3573   Mean   :1638  
##              3rd Qu.: 3134   3rd Qu.:1733  
##              Max.   :48875   Max.   :5975

In the column nCount_RNA, the maximum is far from the third quartile. For visualization purpose, we transform this column to log10 scale.

sobj$log10_nCount_RNA = log10(sobj$nCount_RNA)

summary(sobj@meta.data)
##  orig.ident    nCount_RNA     nFeature_RNA  log10_nCount_RNA
##  TD3A:4587   Min.   :  502   Min.   :  22   Min.   :2.701   
##              1st Qu.: 1986   1st Qu.:1291   1st Qu.:3.298   
##              Median : 2397   Median :1476   Median :3.380   
##              Mean   : 3573   Mean   :1638   Mean   :3.429   
##              3rd Qu.: 3134   3rd Qu.:1733   3rd Qu.:3.496   
##              Max.   :48875   Max.   :5975   Max.   :4.689

We compute the percentage of UMI related for each of the three list of genes. First, we compute the proportion of transcripts related to the genes encoded in the mitochondria, per cell.

sobj = Seurat::PercentageFeatureSet(
  sobj,
  assay = "RNA",
  features = mito_symbols,
  col.name = "percent.mt")

# Alternative way to do (almost) the same:
# sobj = Seurat::PercentageFeatureSet(
#   sobj,
#   assay = "RNA",
#   pattern = "^mt",
#   col.name = "percent.mt")

summary(sobj$percent.mt)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   0.000   1.985   2.489   2.997   3.125  97.250

Then, we compute the proportion of transcripts related to the genes encoding ribosomal units, per cell:

sobj = Seurat::PercentageFeatureSet(
  sobj,
  assay = "RNA",
  features = ribo_symbols,
  col.name = "percent.rb")

# Alternative way to do (almost) the same:
# sobj = Seurat::PercentageFeatureSet(
#   sobj,
#   assay = "RNA",
#   pattern = "^Rp[l|s][a-z]?[0-9]*[a,x]?$",
#   col.name = "percent.rb")

summary(sobj$percent.rb)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   0.000   7.884   9.577  10.880  12.167  42.010

Finally, we compute the proportion of transcripts related to stress signature, per cell:

sobj = Seurat::PercentageFeatureSet(
  sobj,
  assay = "RNA",
  features = stress_symbols,
  col.name = "percent.st")

summary(sobj$percent.st)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   0.000   2.946   3.338   3.368   3.749  10.198

We know have all the QC-related metrics:

summary(sobj@meta.data)
##  orig.ident    nCount_RNA     nFeature_RNA  log10_nCount_RNA   percent.mt    
##  TD3A:4587   Min.   :  502   Min.   :  22   Min.   :2.701    Min.   : 0.000  
##              1st Qu.: 1986   1st Qu.:1291   1st Qu.:3.298    1st Qu.: 1.985  
##              Median : 2397   Median :1476   Median :3.380    Median : 2.489  
##              Mean   : 3573   Mean   :1638   Mean   :3.429    Mean   : 2.997  
##              3rd Qu.: 3134   3rd Qu.:1733   3rd Qu.:3.496    3rd Qu.: 3.125  
##              Max.   :48875   Max.   :5975   Max.   :4.689    Max.   :97.250  
##    percent.rb       percent.st    
##  Min.   : 0.000   Min.   : 0.000  
##  1st Qu.: 7.884   1st Qu.: 2.946  
##  Median : 9.577   Median : 3.338  
##  Mean   :10.880   Mean   : 3.368  
##  3rd Qu.:12.167   3rd Qu.: 3.749  
##  Max.   :42.010   Max.   :10.198

Failing cells

We identify the cells that do not pass the quality control. This will be used for the visualization and for filtering. If the filtering thresholds are modified, do not forget to run again this chunk.

fail_percent.mt = sobj@meta.data %>%
  dplyr::filter(percent.mt > cut_percent.mt) %>%
  rownames()

fail_percent.rb = sobj@meta.data %>%
  dplyr::filter(percent.rb > cut_percent.rb) %>%
  rownames()

fail_percent.st = sobj@meta.data %>%
  dplyr::filter(percent.st > cut_percent.st) %>%
  rownames()

fail_log10_nCount_RNA = sobj@meta.data %>%
  dplyr::filter(log10_nCount_RNA < cut_log10_nCount_RNA) %>%
  rownames()

fail_nFeature_RNA = sobj@meta.data %>%
  dplyr::filter(nFeature_RNA < cut_nFeature_RNA) %>%
  rownames()

Visualization

This is difficult to handle the distribution of these metrics across cells. We opt for various visualization ways:

  • histogram, showing the distribution of the metric
  • violin/box plot, showing the distribution of the metric, useful if several datasets are considered
  • UMAP (or alternatively, tSNE), showing the distribution of the metric over a 2D projection of cells

You may choose one of these visualization ways.

Number of UMI

Define the code

We write a lot of code to create our figures of interest:

# Histogram showing:
# - the distribution of the metric
# - an estimated density
# - the filtering threshold as a straight line
p_hist = ggplot2::ggplot(sobj@meta.data, aes(x = log10_nCount_RNA)) +
  ggplot2::geom_histogram(aes(y = ggplot2::after_stat(density)),
                          colour = "black", fill = "#F8766D", bins = 100) +
  ggplot2::geom_density(alpha = 0, col = "blue", lwd = 0.75) +
  ggplot2::geom_vline(xintercept = cut_log10_nCount_RNA, col = "red") +
  ggplot2::labs(title = paste0("Threshold for log10_nCount_RNA is: ", cut_log10_nCount_RNA)) +
  ggplot2::theme_classic() +
  ggplot2::theme(plot.title = element_text(hjust = 0.5))

# Violin plot:
# - is easily accessible in the Seurat package
# - can or not display the cells (set `pt.size = 0` to hide the cells)
# - is useful when several datasets have been merged
p_violin = Seurat::VlnPlot(sobj, features = "log10_nCount_RNA") +
  ggplot2::geom_hline(yintercept = cut_log10_nCount_RNA, col = "red") +
  ggplot2::theme(axis.title.x = element_blank(),
                 legend.position = "none")

# Box plot is similar to violin plot but not in the Seurat package
p_boxplot = ggplot2::ggplot(sobj@meta.data, aes(y = log10_nCount_RNA,
                                                x = orig.ident)) +
  ggplot2::geom_boxplot(colour = "black", fill = "#F8766D") +
  ggplot2::geom_jitter(width = 0.3, size = 0.001) +
  ggplot2::geom_hline(yintercept = cut_log10_nCount_RNA, col = "red") +
  ggplot2::theme_classic() +
  ggplot2::theme(axis.title.x = element_blank())

# This 2D representation shows the distribution of the metric over cells
p_umap = Seurat::FeaturePlot(sobj,
                             reduction = "umap",
                             features = "log10_nCount_RNA") +
  ggplot2::scale_color_gradientn(colors = color_palette) +
  ggplot2::theme(aspect.ratio = 1)

# This 2D representation shows the low quality cells in color
# `order = "fail"` is used to display failing cells on front
# before, we need to define the "failorpass" column in sobj@meta.data
# It corresponds to "fail" for cells that do no pass the threshold,
# or "pass" otherwise
sobj$failorpass = ifelse(colnames(sobj) %in% fail_log10_nCount_RNA,
                         yes = "fail", no = "pass") %>%
  as.factor()

p_fail = Seurat::DimPlot(sobj,
                         group.by = "failorpass",
                         order = "fail") +
  ggplot2::scale_color_manual(values = c("#F8766D", "gray80"),
                              breaks = levels(sobj$failorpass)) +
  ggplot2::labs(title = "log10_nCount_RNA",
                subtitle = paste0(length(fail_log10_nCount_RNA), " cells fail (",
                                  round(100*length(fail_log10_nCount_RNA)/ncol(sobj), 2), " %)")) +
  ggplot2::theme(aspect.ratio = 1,
                 plot.title = element_text(hjust = 0.5),
                 plot.subtitle = element_text(hjust = 0.5))

sobj$failorpass = NULL # remove the column (it was temporary)

# We use the patchwork package to arrange all figures together
patchwork::wrap_plots(p_umap, p_fail, p_hist, p_violin, p_boxplot) +
  patchwork::plot_layout(nrow = 1, widths = c(1, 1, 2, 1, 1))

Define a function

Instead of copying-pasting this section of code for all QC-metrics, we design a function using the template below:

my_function_name = function(param1, param2) {
  # do something with the parameter values
  output = "something"
  
  return(output)
}

Here is the function:

print_1_qc_metric = function(object = sobj,
                             qc = "log10_nCount_RNA",
                             cut_qc = cut_log10_nCount_RNA,
                             failing_cells = fail_log10_nCount_RNA) {
  # Description of the parameters:
  # - sobj : the Seurat object, with default value to the one
  # - qc : CHARACTER : the QC metric, must be a column in sobj@meta.data
  # - cut_qc : NUMERIC : the filtering threshold for the QC metric
  # - failing_cells : CHARACTER VECTOR : the cells that fail the QC
  
  # Histogram
  p_hist = ggplot2::ggplot(object@meta.data, aes(x = .data[[qc]])) +
    ggplot2::geom_histogram(aes(y = ggplot2::after_stat(density)),
                            colour = "black", fill = "#F8766D", bins = 100) +
    ggplot2::geom_density(alpha = 0, col = "blue", lwd = 0.75) +
    ggplot2::geom_vline(xintercept = cut_qc, col = "red") +
    ggplot2::labs(title = paste0("Threshold for ", qc, " is: ", cut_qc)) +
    ggplot2::theme_classic() +
    ggplot2::theme(plot.title = element_text(hjust = 0.5))
  
  # Violin plot
  p_violin = Seurat::VlnPlot(object, features = qc) +
    ggplot2::geom_hline(yintercept = cut_qc, col = "red") +
    ggplot2::theme(axis.title.x = element_blank(),
                   legend.position = "none")
  
  # Box plot
  p_boxplot = ggplot2::ggplot(object@meta.data, aes(y = .data[[qc]],
                                                    x = "orig.ident")) +
    ggplot2::geom_boxplot(colour = "black", fill = "#F8766D") +
    ggplot2::geom_jitter(width = 0.3, size = 0.001) +
    ggplot2::geom_hline(yintercept = cut_qc, col = "red") +
    ggplot2::theme_classic() +
    ggplot2::theme(axis.title.x = element_blank())
  
  # Feature plot
  p_umap = Seurat::FeaturePlot(object,
                               reduction = "umap",
                               features = qc) +
    ggplot2::scale_color_gradientn(colors = color_palette) +
    ggplot2::theme(aspect.ratio = 1)
  
  # Dim plot
  object$failorpass = ifelse(colnames(object) %in% failing_cells,
                             yes = "fail", no = "pass") %>%
    as.factor()
  
  p_fail = Seurat::DimPlot(object,
                           group.by = "failorpass",
                           order = "fail") +
    ggplot2::scale_color_manual(values = c("#F8766D", "gray80"),
                                breaks = levels(object$failorpass)) +
    ggplot2::labs(title = qc,
                  subtitle = paste0(length(failing_cells), " cells fail (",
                                    round(100*length(failing_cells)/ncol(sobj), 2), " %)")) +
    ggplot2::theme(aspect.ratio = 1,
                   plot.title = element_text(hjust = 0.5),
                   plot.subtitle = element_text(hjust = 0.5))
  
  # Patchwork
  p = patchwork::wrap_plots(p_umap, p_fail, p_hist, p_violin, p_boxplot) +
    patchwork::plot_layout(nrow = 1, widths = c(1, 1, 2, 1, 1))
  
  return(p)
}

Check the function

This is working for log10_nCount_RNA, because it is used to define default parameter values:

print_1_qc_metric()

but also works by specifying the parameter values:

print_1_qc_metric(sobj,
                  qc = "log10_nCount_RNA",
                  cut_qc = cut_log10_nCount_RNA,
                  failing_cells = fail_log10_nCount_RNA)

So we can copy-paste only this chunk for the next QC metrics !

Number of genes

print_1_qc_metric(sobj,
                  qc = "nFeature_RNA",
                  cut_qc = cut_nFeature_RNA,
                  failing_cells = fail_nFeature_RNA)

Mitochondrial genes expression

print_1_qc_metric(sobj,
                  qc = "percent.mt",
                  cut_qc = cut_percent.mt,
                  failing_cells = fail_percent.mt)

Ribosomal genes expression

print_1_qc_metric(sobj,
                  qc = "percent.rb",
                  cut_qc = cut_percent.rb,
                  failing_cells = fail_percent.rb)

Filtering

We could filter out cells based on these 5 QC metrics now. It is also possible to wait, perform various annotations such as cell type annotation or cell cycle phase scoring, to better characterize the low quality cells.

To filter a Seurat object, we use the subset function:

sobj_filtered = subset(sobj, invert = TRUE,
                       cells = unique(c(fail_log10_nCount_RNA, fail_nFeature_RNA,
                                        fail_percent.mt, fail_percent.rb, fail_percent.st)))
sobj_filtered
## An object of class Seurat 
## 31053 features across 4216 samples within 1 assay 
## Active assay: RNA (31053 features, 2000 variable features)
##  3 layers present: counts, data, scale.data
##  2 dimensional reductions calculated: pca, umap

We are going to save the object annotated for cells that fail the quality control, regardless the metrics. So we add a single column to sobj@meta.data :

sobj$fail_qc = ifelse(colnames(sobj) %in% c(fail_log10_nCount_RNA, fail_nFeature_RNA,
                                            fail_percent.mt, fail_percent.rb, fail_percent.st),
                      yes = "fail", no = "pass")

table(sobj$fail_qc)
## 
## fail pass 
##  371 4216

Save

We save the non-filtered Seurat object:

saveRDS(sobj, file = paste0(outdir, "/sobj_TD3A_qc_annotated.rds"))

This Seurat object can then be the input object for annotation, definition of a new projection and downstream analyses.

R session

This is a good practice to show the version of the packages used in this notebook.

sessionInfo()
## R version 4.4.1 (2024-06-14)
## Platform: x86_64-conda-linux-gnu
## Running under: Ubuntu 20.04.6 LTS
## 
## Matrix products: default
## BLAS/LAPACK: /shared/ifbstor1/software/miniconda/envs/r-4.4.1/lib/libopenblasp-r0.3.27.so;  LAPACK version 3.12.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Europe/Paris
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] ggplot2_3.5.1 dplyr_1.1.4  
## 
## loaded via a namespace (and not attached):
##   [1] deldir_2.0-4           pbapply_1.7-2          gridExtra_2.3         
##   [4] rlang_1.1.4            magrittr_2.0.3         RcppAnnoy_0.0.22      
##   [7] spatstat.geom_3.3-3    matrixStats_1.4.1      ggridges_0.5.6        
##  [10] compiler_4.4.1         png_0.1-8              vctrs_0.6.5           
##  [13] reshape2_1.4.4         stringr_1.5.1          pkgconfig_2.0.3       
##  [16] fastmap_1.2.0          labeling_0.4.3         utf8_1.2.4            
##  [19] promises_1.3.0         rmarkdown_2.28         ggbeeswarm_0.7.2      
##  [22] purrr_1.0.2            xfun_0.48              cachem_1.1.0          
##  [25] jsonlite_1.8.9         goftest_1.2-3          highr_0.11            
##  [28] later_1.3.2            spatstat.utils_3.1-0   irlba_2.3.5.1         
##  [31] parallel_4.4.1         cluster_2.1.6          R6_2.5.1              
##  [34] ica_1.0-3              spatstat.data_3.1-2    bslib_0.8.0           
##  [37] stringi_1.8.4          RColorBrewer_1.1-3     reticulate_1.39.0     
##  [40] spatstat.univar_3.0-1  parallelly_1.38.0      lmtest_0.9-40         
##  [43] jquerylib_0.1.4        scattermore_1.2        Rcpp_1.0.13           
##  [46] knitr_1.48             tensor_1.5             future.apply_1.11.2   
##  [49] zoo_1.8-12             sctransform_0.4.1      httpuv_1.6.15         
##  [52] Matrix_1.7-1           splines_4.4.1          igraph_2.1.1          
##  [55] tidyselect_1.2.1       abind_1.4-8            rstudioapi_0.17.0     
##  [58] yaml_2.3.10            spatstat.random_3.3-2  spatstat.explore_3.3-2
##  [61] codetools_0.2-20       miniUI_0.1.1.1         listenv_0.9.1         
##  [64] lattice_0.22-6         tibble_3.2.1           plyr_1.8.9            
##  [67] shiny_1.9.1            withr_3.0.1            ROCR_1.0-11           
##  [70] ggrastr_1.0.2          evaluate_1.0.1         Rtsne_0.17            
##  [73] future_1.34.0          fastDummies_1.7.4      survival_3.7-0        
##  [76] polyclip_1.10-7        fitdistrplus_1.2-1     pillar_1.9.0          
##  [79] Seurat_5.1.0           KernSmooth_2.23-24     plotly_4.10.4         
##  [82] generics_0.1.3         RcppHNSW_0.6.0         sp_2.1-4              
##  [85] munsell_0.5.1          scales_1.3.0           globals_0.16.3        
##  [88] xtable_1.8-4           glue_1.8.0             lazyeval_0.2.2        
##  [91] tools_4.4.1            data.table_1.16.2      RSpectra_0.16-2       
##  [94] RANN_2.6.2             leiden_0.4.3.1         dotCall64_1.2         
##  [97] cowplot_1.1.3          grid_4.4.1             tidyr_1.3.1           
## [100] colorspace_2.1-1       nlme_3.1-165           patchwork_1.3.0       
## [103] beeswarm_0.4.0         vipor_0.4.7            cli_3.6.3             
## [106] spatstat.sparse_3.1-0  spam_2.11-0            fansi_1.0.6           
## [109] viridisLite_0.4.2      uwot_0.2.2             gtable_0.3.5          
## [112] sass_0.4.9             digest_0.6.37          progressr_0.14.0      
## [115] ggrepel_0.9.6          htmlwidgets_1.6.4      SeuratObject_5.0.2    
## [118] farver_2.1.2           htmltools_0.5.8.1      lifecycle_1.0.4       
## [121] httr_1.4.7             mime_0.12              MASS_7.3-61
LS0tCnRpdGxlOiAiUHJlLXByb2Nlc3Npbmc6IHF1YWxpdHkgY29udHJvbCIKc3VidGl0bGU6ICJFQkFJSSBuMSAyMDI0OiBzY1JOQS1TZXEgd29ya3Nob3AiCmRhdGU6ICIyMDI0LTExLTE5IgpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDogCiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgY29kZV9kb3dubG9hZDogdHJ1ZQotLS0KCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUsCiAgICAgICAgICAgICAgICAgICAgICBtZXNzYWdlID0gRkFMU0UpCmBgYAoKKipEZXNjcmlwdGlvbioqOgoKVGhpcyBmaWxlIGRlc2NyaWJlcyB0aGUgZGlmZmVyZW50IHN0ZXBzIHRvIHRoZSBxdWFsaXR5IGNvbnRyb2wgYmFzZWQgb246CgoqIHRoZSBudW1iZXIgb2YgVU1JICh0cmFuc2NyaXB0cykgZGV0ZWN0ZWQgcGVyIGNlbGwKKiB0aGUgbnVtYmVyIG9mIGdlbmVzIGRldGVjdGVkIHBlciBjZWxsCiogdGhlIHByb3BvcnRpb24gb2YgdHJhbnNjcmlwdHMgcmVsYXRlZCB0byB0aGUgZ2VuZXMgZW5jb2RlZCBpbiB0aGUgbWl0b2Nob25kcmlhLCBwZXIgY2VsbAoqIHRoZSBwcm9wb3J0aW9uIG9mIHRyYW5zY3JpcHRzIHJlbGF0ZWQgdG8gdGhlIGdlbmVzIGVuY29kaW5nIHJpYm9zb21hbCB1bml0cywgcGVyIGNlbGwKKiB0aGUgcHJvcG9ydGlvbiBvZiB0cmFuc2NyaXB0cyByZWxhdGVkIHRvIHN0cmVzcyBzaWduYXR1cmUsIHBlciBjZWxsCgoqKklucHV0IGRhdGEqKjogU2V1cmF0IG9iamVjdCBjb250YWluaW5nIGFsbCBjZWxscyAoZmlsdGVyZWQgY291bnQgbWF0cml4KQoKKipPdXRwdXQgZGF0YSoqOiBTZXVyYXQgb2JqZWN0IGFubm90YXRlZCBmb3IgdGhlIGNlbGxzIHRvIGZpbHRlciBvdXQgZm9yIGRvd25zdHJlYW0gYW5hbHlzaXMsIGNhbGxlZCBgc29ial9URDNBX3FjX2Fubm90YXRlZC5yZHNgCgoqKk5leHQgc3RlcHMqKjogQ2VsbHMgYW5ub3RhdGlvbiBmb3IgY2VsbCB0eXBlLCBkb3VibGV0IHN0YXR1cyBhbmQgY2VsbCBjeWNsZSBzdGF0dXMKCiMgQ29udGV4dAoKSW4gdGhpcyBmaWxlLCB3ZSB1c2VkIGEgZGF0YXNldCBmcm9tIHRoZSBbUGFpdmEgZXQgYWwuXShodHRwczovL2ZlYnMub25saW5lbGlicmFyeS53aWxleS5jb20vZG9pL2Z1bGwvMTAuMTExMS9mZWJzLjE0NjUxKSBwdWJsaWNhdGlvbi4KClRoZSBzdHVkeSBjb25jZXJucyAqKnRoeW11cyBhdXRvbm9teSoqOgoKKiBUaGUgdGh5bXVzIGlzIGFuICJvcmdhbiBvZiBwYXNzYWdlIiwgY3JpdGljYWwgaW4gaXRzIGZ1bmN0aW9uIHRvIHRoZSBhZGFwdGl2ZSBpbW11bmUgc3lzdGVtIGZvciB0aGUgbWF0dXJhdGlvbiBvZiBUIGNlbGwgbHltcGhvY3l0ZXMuCiogVGhpcyBtYXR1cmF0aW9uIGludm9sdmVzIHR3byBtYWluIHN0ZXBzLCBwZXJmb3JtZWQgdGhhbmtzIHRvIG1hY3JvcGhhZ2VzOgogICogUG9zaXRpdmUgc2VsZWN0aW9uIDoga2VlcGluZyBjZWxscyB0aGF0IHN1Y2Nlc3NmdWxseSBkZXZlbG9wIHJlYWN0IGFwcHJvcHJpYXRlbHkgd2l0aCBNSEMgaW1tdW5lIHJlY2VwdG9ycyBvZiB0aGUgYm9keQogICogTmVnYXRpdmUgc2VsZWN0aW9uIDoga2VlcGluZyBjZWxscyB0aGF0IGRvIG5vdCByZWFjdCBhZ2FpbnN0IG5hdHVyYWwgcHJvdGVpbnMgb2YgdGhlIGJvZHkuCiogVGh5bXVzIF9hdXRvbm9teV8gaXMgYSBuYXR1cmFsIG1lY2hhbmlzbSB0aGF0IGFsbG93cyB0byBjcmVhdGUgVCBjZWxscyBpbiB0aGUgdGh5bXVzIGJ5IGRpZmZlcmVudGlhdGlvbiBhbmQgY2VsbCBjb21wZXRpdGlvbiwgZXZlbiB3aGVuIG5vcm1hbCBwcm9nZW5pdG9ycyBmcm9tIHRoZSBib25lIG1hcnJvdyBhcmUgbGFja2luZywgaW4gY3JpdGljYWwgY29uZGl0aW9ucy4KKiBUaGlzIG1lY2hhbmlzbSBpcyBrbm93biBpbiBpdHMgZWZmZWN0cywgYnV0IHRoZSBjZWxscyBpbnZvbHZlZCBpbiBhcmUgbm90LgoqIFRoaXMgc3R1ZHkgaXMgb2YgaW1wb3J0YW5jZSBpbiB0aGUgaGVhbHRoIGZpZWxkLCBhcyB0aGlzIG1lY2hhbmlzbSByZWxpZXMgb24gYSB0ZW1wb3JhcnkgbG9zcyBvZiBjb250cm9sIG9mIHRoZSBjZWxsIG5vcm1hbCBmdW5jdGlvbnMuCiogVGhlIGNvbnNlcXVlbmNlIGlzIHRoYXQgaWYgdGh5bXVzIGlzIGluIGF1dG9ub215IGZvciB0b28gbG9uZyAoZmV3IHdlZWtzKSwgdGhpcyBpcyBhIHByZWx1ZGUgZm9yIGxldWtlbWlhICEKCiAgPGNlbnRlcj4hW10odGh5bXVzX2F1dG9ub215LnBuZyk8L2NlbnRlcj4KCiogT3JnYW5pc20gaXMgOiAqKm11cyBtdXNjdWx1cyoqCiogSW5kaXZpZHVhbHMgYXJlIDogbWljZSAqKmluIGRldmVsb3BtZW50LCBncmFmdGVkKioKKiBUaGUgZGVzaWduIGNvcnJlc3BvbmRzIHRvICoqdHdvIGNvbmRpdGlvbnMqKiAoVGVzdCAvIGNvbnRyb2wpCiAgKiBDb250cm9sIDogdGh5bXVzIGZyb20gKip3aWxkIHR5cGUqKiBuZXdib3JuIG1vdXNlIHRyYW5zcGxhbnRlZCBpbnRvICoqd2lsZCB0eXBlKioganV2ZW5pbGUgbW91c2UuIEluIHRoaXMgY29udHJvbCBjYXNlLCAqKmRvbm9yKiogVC1jZWxscyBwcm9nZW5pdG9ycyAoRE4zKSB3ZXJlIHJlcGxhY2VkIGJ5ICoqaG9zdCoqIGNlbGxzICoqMyB3ZWVrcyoqIGFmdGVyIHRyYW5zcGxhbnRhdGlvbi4KICAqIFRlc3QgOiB0aHltdXMgZnJvbSAqKndpbGQgdHlwZSoqIG5ld2Jvcm4gbW91c2UgdHJhbnNwbGFudGVkIGluICoqS08gUmFnLS8tKiogdHlwZSBqdXZlbmlsZSBtb3VzZSAodGhlIEtPIHBhcnRpYWxseSBpbXBhaXJzIHRoZWlyIGFiaWxpdHkgdG8gcHJvZHVjZSBULWNlbGwgcHJvZ2VuaXRvcnMgaW4gbm9ybWFsIGFtb3VudHMpLiBJbiB0aGlzIHRlc3QgY2FzZSwgKipkb25vcioqIFQtY2VsbHMgcHJvZ2VuaXRvcnMgKEROMykgd2VyZSByZXBsYWNlZCBieSAqKmhvc3QqKiBjZWxscyAqKjkgd2Vla3MqKiBhZnRlciB0cmFuc3BsYW50YXRpb24sIHNob3dpbmcgdGhhdCB0aGUgZG9ub3IgRE4zIGNlbGxzIG91dGxpdmVkIHRoZWlyIG5vcm1hbCBsaWZlc3BhbiBieSB+NiB3ZWVrcy4KICAKICA8Y2VudGVyPiFbXShwYWl2YV93dF9rby5wbmcpPC9jZW50ZXI+CiAgCgpZb3Ugd2lsbCBtYWlubHkgd29yayBvbiB0aGUgKipLTyBzYW1wbGUgKCdURDNBJykqKi4gVGhlIGlucHV0IGRhdGEgY29uc2lzdHMgaW4gYSAqKmNvdW50IG1hdHJpeCoqLCBhcyBhIGd6aXBwZWQgdGFidWxhciB0ZXh0IGZpbGUsIHRoYXQgY29udGFpbnMgZXZlcnl0aGluZyBuZWVkZWQgdG8gY3JlYXRlIGEgYmFzaWMgU2V1cmF0IG9iamVjdCA6CiAgKiBUaGUgZXhwcmVzc2lvbnMgKipjb3VudHMqKgogICogVGhlICoqZmVhdHVyZSBuYW1lcyoqIChoZXJlLCBnZW5lIHN5bWJvbHMpCiAgKiBUaGUgKipiYXJjb2RlIG5hbWVzKioKClRoaXMgbWF0cml4IGhhcyBhbHJlYWR5IGJlZW4gKipmaWx0ZXJlZCBmb3IgZW1wdHkgZHJvcGxldHMqKi4KCiMgUHJlLXJlcXVpc2l0ZXMKClRvIG9idGFpbiB0aGlzIGZpbGUgYW5kIHN0YXJ0cyB0aGUgVEQsIGNvcHktcGFzdGUgdGhlIGZvbGRlciBpbiB5b3VyIHByb2plY3QsIGVpdGhlciB1c2luZyB0aGUgZ3JhcGhpY2FsIGludGVyZmFjZSwgb3IgaW4gdGhlIFRlcm1pbmFsOgoKYGBge2Jhc2gsIGV2YWw9RkFMU0V9CmNwIC1yIC9zaGFyZWQvcHJvamVjdHMvMjQyMl9lYmFpaV9uMS9hdGVsaWVyX3Njcm5hc2VxL1JNRC9QcmVwcm9jLjIgL3NoYXJlZC9wcm9qZWN0cy9beW91cl9wcm9qZWN0c10vVERfUHJlcHJvYy4yCgpjZCAvc2hhcmVkL3Byb2plY3RzL1t5b3VyX3Byb2plY3RzXS9URF9QcmVwcm9jLjIKYGBgCgpUaGUgdHJlZSBzaG91bGQgbG9va3MgbGlrZToKCmBgYHtiYXNofQpscwpgYGAKCiMgRW52aXJvbm1lbnQKCldlIGxvYWQgdGhlIHBhY2thZ2Ugb2YgaW50ZXJlc3Q6CgpgYGB7ciBwYWNrYWdlcywgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KbGlicmFyeShkcGx5cikKbGlicmFyeShnZ3Bsb3QyKQoKLmxpYlBhdGhzKCkKYGBgCgojIERhdGEKCldlIGxvYWQgdGhlIGNvdW50IG1hdHJpeDoKCmBgYHtyIGxvYWRfY291bnRfbWF0cml4fQojIFNldCBpbnB1dCBkYXRhCmlucHV0X2RhdGEgPSAiLi9kYXRhL0dTTTQ4NjExOTRfZ2V4XzJfcmF3X2dlbmVfZXhwcmVzc2lvbi50c3YuZ3oiCgojIExvYWQgZGF0YQptYXQgPSByZWFkLnRhYmxlKGlucHV0X2RhdGEsCiAgICAgICAgICAgICAgICAgaGVhZGVyID0gVFJVRSwKICAgICAgICAgICAgICAgICBzZXAgPSAiXHQiKQptYXQgPSBhcy5tYXRyaXgobWF0KQptYXQgPSBNYXRyaXg6Ok1hdHJpeChtYXQsCiAgICAgICAgICAgICAgICAgICAgIHNwYXJzZSA9IFRSVUUpCgpkaW0obWF0KQptYXRbYygxOjUpLCBjKDE6NSldCmBgYAoKV2UgYnVpbGQgYSBTZXVyYXQgb2JqZWN0IGZyb20gdGhlIGNvdW50IG1hdHJpeDoKCmBgYHtyIG1ha2Vfc29ian0Kc29iaiA9IFNldXJhdDo6Q3JlYXRlU2V1cmF0T2JqZWN0KGNvdW50cyA9IG1hdCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzc2F5ID0gIlJOQSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcm9qZWN0ID0gIlREM0EiKQpzb2JqCmBgYAoKV2UgZG8gbm90IG5lZWQgdGhlIGNvdW50IG1hdHJpeDoKCmBgYHtyIHJtX21hdH0Kcm0obWF0KQpgYGAKCgojIEdsb2JhbCBzZXR0aW5ncwoKV2UgZGVmaW5lIHRoZSBvdXRwdXQgZGlyZWN0b3J5IHRvIHNhdmUgdGhlIFNldXJhdCBvYmplY3QgaW4gdGhpcyBlbmQgb2YgdGhpcyBmaWxlOgoKYGBge3Igb3V0ZGlyfQpvdXRkaXIgPSAiLi9kYXRhIgoKaWYgKCFkaXIuZXhpc3RzKG91dGRpcikpIHsKICBkaXIuY3JlYXRlKG91dGRpciwgcmVjdXJzaXZlID0gRkFMU0UpCn0KYGBgCgpXZSBzZXQgdGhlIGZpbHRlcmluZyB0aHJlc2hvbGRzIGJhc2VkIG9uIHF1YWxpdHkgY29udHJvbC1yZWxhdGVkIG1ldHJpY3MuIEFkanVzdCB0aGVtIGFzIG5lY2Vzc2FyeSBiYXNlZCBvbiB0aGUgZmlndXJlcy4KCmBgYHtyIHNldF90aHJlc2hvbGRzfQpjdXRfbG9nMTBfbkNvdW50X1JOQSA9IDMKY3V0X25GZWF0dXJlX1JOQSA9IDc1MApjdXRfcGVyY2VudC5tdCA9IDUKY3V0X3BlcmNlbnQucmIgPSAzMApjdXRfcGVyY2VudC5zdCA9IDYKYGBgCgpXZSBkZWZpbmUgYSBuaWNlIHBhbGV0dGUgdG8gdmlzdWFsaXplIHRoZSBRQyBtZXRyaWNzOgoKYGBge3IgY29sb3JfcGFsZXR0ZX0KY29sb3JfcGFsZXR0ZSA9IGMoImxpZ2h0Z3JheSIsICIjRkRCQjg0IiwgIiNFRjY1NDgiLCAiIzdGMDAwMCIsICJibGFjayIpCmBgYAoKCkZvciB0aGUgUUMgbWV0cmljcyByZWxhdGVkIHRvIHRoZSBwcm9wb3J0aW9uIG9mIFVNSSBiZWxvbmdzIHRvIGEgc3BlY2lmaWMgZ2VuZSBzZXRzLCB3ZSBuZWVkIHRvIGxvYWQgdGhlIGdlbmUgc2V0cy4KCmBgYHtyIGxvYWRfbGlzdHN9Cm1pdG9fc3ltYm9scyA9IHJlYWRSRFMoIi4vcmVzb3VyY2VzL211c19tdXNjdWx1c19taXRvX3N5bWJvbHNfMjAxOTEwMTUucmRzIikKbWl0b19zeW1ib2xzCgpyaWJvX3N5bWJvbHMgPSByZWFkUkRTKCIuL3Jlc291cmNlcy9tdXNfbXVzY3VsdXNfY3JpYm9fc3ltYm9sc18yMDE5MTAxNS5yZHMiKQpyaWJvX3N5bWJvbHMKCnN0cmVzc19zeW1ib2xzID0gcmVhZFJEUygiLi9yZXNvdXJjZXMvbXVzX211c2N1bHVzX3N0cmVzc19zeW1ib2xzXzIwMjAwMjI0LnJkcyIpCnN0cmVzc19zeW1ib2xzCmBgYAoKV2Uga2VlcCBvbmx5IHRoZSBnZW5lIHN5bWJvbHMgYXZhaWxhYmxlIGluIG91ciBkYXRhOgoKYGBge3Igc3Vic2V0X3N5bWJvbHN9Cm1pdG9fc3ltYm9scyA9IGludGVyc2VjdChtaXRvX3N5bWJvbHMsIHJvd25hbWVzKHNvYmopKQptaXRvX3N5bWJvbHMKCnJpYm9fc3ltYm9scyA9IGludGVyc2VjdChyaWJvX3N5bWJvbHMsIHJvd25hbWVzKHNvYmopKQpyaWJvX3N5bWJvbHMKCnN0cmVzc19zeW1ib2xzID0gaW50ZXJzZWN0KHN0cmVzc19zeW1ib2xzLCByb3duYW1lcyhzb2JqKSkKc3RyZXNzX3N5bWJvbHMKYGBgCgoqKk5vdGUqKjogQSBtb3JlIHJvYnVzdCBhbmFseXNpcyBtYXkgdXNlIHRoZSBnZW5lIGlkZW50aWZpZXJzIGluc3RlYWQgb2YgZ2VuZSBzeW1ib2xzLgoKIyBGYXN0LXByb2Nlc3NpbmcKClRvIHZpc3VhbGl6ZSB0aGUgbG93IHF1YWxpdHkgY2VsbHMsIHdlIGdlbmVyYXRlIGEgcHJvamVjdGlvbi4gVW5kZXJzdGFuZGluZyB0aGUgY29tbWFuZHMgYmVsb3cgaXMgdGhlIHB1cnBvc2Ugb2YgbmV4dCBjb3Vyc2Ugc2Vzc2lvbnMuIFNvIGp1c3QgcnVuOgoKYGBge3IgcXVpY2tfcHJvY2Vzc2luZywgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0Kc29iaiA9IFNldXJhdDo6Tm9ybWFsaXplRGF0YShzb2JqKQpzb2JqID0gU2V1cmF0OjpTY2FsZURhdGEoc29iaikKc29iaiA9IFNldXJhdDo6RmluZFZhcmlhYmxlRmVhdHVyZXMoc29iaikKc29iaiA9IFNldXJhdDo6UnVuUENBKHNvYmopCnNvYmogPSBTZXVyYXQ6OlJ1blVNQVAoc29iaiwgZGltcyA9IGMoMToyMCkpCgpzb2JqCmBgYAoKV2UgY2FuIG5vdyB2aXN1YWxpemUgdGhlIGNlbGxzIG9uIGEgMkQgcHJvamVjdGlvbjoKCmBgYHtyIHZpc3VhbGl6ZV9jZWxscywgZmlnLndpZHRoID0gOCwgZmlnLmhlaWdodCA9IDh9ClNldXJhdDo6RGltUGxvdChzb2JqLAogICAgICAgICAgICAgICAgcmVkdWN0aW9uID0gInVtYXAiLAogICAgICAgICAgICAgICAgY29scyA9ICJibGFjayIpICsKICBnZ3Bsb3QyOjp0aGVtZShhc3BlY3QucmF0aW8gPSAxLAogICAgICAgICAgICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikKYGBgCgojIFF1YWxpdHkgY29udHJvbAoKIyMgQ29tcHV0ZSBtZXRyaWNzCgpXaGF0IGlzIGFscmVhZHkgYXZhaWxhYmxlIGluIHRoZSBTZXVyYXQgb2JqZWN0ID8KCmBgYHtyIHNlZV9tZXRhZGF0YX0KaGVhZChzb2JqQG1ldGEuZGF0YSkKYGBgCgpIb3cgZG8gdGhlIHR3byBmaXJzdCBRQyBtZXRyaWNzIHZhcnkgPwoKYGBge3Igc3VtbWFyeV9tZXRhZGF0YX0Kc3VtbWFyeShzb2JqQG1ldGEuZGF0YSkKYGBgCgpJbiB0aGUgY29sdW1uIGBuQ291bnRfUk5BYCwgdGhlIG1heGltdW0gaXMgZmFyIGZyb20gdGhlIHRoaXJkIHF1YXJ0aWxlLiBGb3IgdmlzdWFsaXphdGlvbiBwdXJwb3NlLCB3ZSB0cmFuc2Zvcm0gdGhpcyBjb2x1bW4gdG8gbG9nMTAgc2NhbGUuCgpgYGB7ciBsb2cxMF9uY291bnRfUk5BfQpzb2JqJGxvZzEwX25Db3VudF9STkEgPSBsb2cxMChzb2JqJG5Db3VudF9STkEpCgpzdW1tYXJ5KHNvYmpAbWV0YS5kYXRhKQpgYGAKCldlIGNvbXB1dGUgdGhlIHBlcmNlbnRhZ2Ugb2YgVU1JIHJlbGF0ZWQgZm9yIGVhY2ggb2YgdGhlIHRocmVlIGxpc3Qgb2YgZ2VuZXMuIEZpcnN0LCB3ZSBjb21wdXRlIHRoZSBwcm9wb3J0aW9uIG9mIHRyYW5zY3JpcHRzIHJlbGF0ZWQgdG8gdGhlIGdlbmVzIGVuY29kZWQgaW4gdGhlIG1pdG9jaG9uZHJpYSwgcGVyIGNlbGwuCgpgYGB7ciBwZXJjZW50X210fQpzb2JqID0gU2V1cmF0OjpQZXJjZW50YWdlRmVhdHVyZVNldCgKICBzb2JqLAogIGFzc2F5ID0gIlJOQSIsCiAgZmVhdHVyZXMgPSBtaXRvX3N5bWJvbHMsCiAgY29sLm5hbWUgPSAicGVyY2VudC5tdCIpCgojIEFsdGVybmF0aXZlIHdheSB0byBkbyAoYWxtb3N0KSB0aGUgc2FtZToKIyBzb2JqID0gU2V1cmF0OjpQZXJjZW50YWdlRmVhdHVyZVNldCgKIyAgIHNvYmosCiMgICBhc3NheSA9ICJSTkEiLAojICAgcGF0dGVybiA9ICJebXQiLAojICAgY29sLm5hbWUgPSAicGVyY2VudC5tdCIpCgpzdW1tYXJ5KHNvYmokcGVyY2VudC5tdCkKYGBgCgpUaGVuLCB3ZSBjb21wdXRlIHRoZSBwcm9wb3J0aW9uIG9mIHRyYW5zY3JpcHRzIHJlbGF0ZWQgdG8gdGhlIGdlbmVzIGVuY29kaW5nIHJpYm9zb21hbCB1bml0cywgcGVyIGNlbGw6CgpgYGB7ciBwZXJjZW50X3JifQpzb2JqID0gU2V1cmF0OjpQZXJjZW50YWdlRmVhdHVyZVNldCgKICBzb2JqLAogIGFzc2F5ID0gIlJOQSIsCiAgZmVhdHVyZXMgPSByaWJvX3N5bWJvbHMsCiAgY29sLm5hbWUgPSAicGVyY2VudC5yYiIpCgojIEFsdGVybmF0aXZlIHdheSB0byBkbyAoYWxtb3N0KSB0aGUgc2FtZToKIyBzb2JqID0gU2V1cmF0OjpQZXJjZW50YWdlRmVhdHVyZVNldCgKIyAgIHNvYmosCiMgICBhc3NheSA9ICJSTkEiLAojICAgcGF0dGVybiA9ICJeUnBbbHxzXVthLXpdP1swLTldKlthLHhdPyQiLAojICAgY29sLm5hbWUgPSAicGVyY2VudC5yYiIpCgpzdW1tYXJ5KHNvYmokcGVyY2VudC5yYikKYGBgCgoKRmluYWxseSwgd2UgY29tcHV0ZSB0aGUgcHJvcG9ydGlvbiBvZiB0cmFuc2NyaXB0cyByZWxhdGVkIHRvIHN0cmVzcyBzaWduYXR1cmUsIHBlciBjZWxsOgoKYGBge3IgcGVyY2VudF9zdH0Kc29iaiA9IFNldXJhdDo6UGVyY2VudGFnZUZlYXR1cmVTZXQoCiAgc29iaiwKICBhc3NheSA9ICJSTkEiLAogIGZlYXR1cmVzID0gc3RyZXNzX3N5bWJvbHMsCiAgY29sLm5hbWUgPSAicGVyY2VudC5zdCIpCgpzdW1tYXJ5KHNvYmokcGVyY2VudC5zdCkKYGBgCgoKV2Uga25vdyBoYXZlIGFsbCB0aGUgUUMtcmVsYXRlZCBtZXRyaWNzOgoKYGBge3IgZmluYWxfc3VtbWFyeX0Kc3VtbWFyeShzb2JqQG1ldGEuZGF0YSkKYGBgCgojIyBGYWlsaW5nIGNlbGxzCgpXZSBpZGVudGlmeSB0aGUgY2VsbHMgdGhhdCBkbyBub3QgcGFzcyB0aGUgcXVhbGl0eSBjb250cm9sLiBUaGlzIHdpbGwgYmUgdXNlZCBmb3IgdGhlIHZpc3VhbGl6YXRpb24gYW5kIGZvciBmaWx0ZXJpbmcuIElmIHRoZSBmaWx0ZXJpbmcgdGhyZXNob2xkcyBhcmUgbW9kaWZpZWQsIGRvIG5vdCBmb3JnZXQgdG8gcnVuIGFnYWluIHRoaXMgY2h1bmsuCgpgYGB7ciBmYWlsX2NlbGxzfQpmYWlsX3BlcmNlbnQubXQgPSBzb2JqQG1ldGEuZGF0YSAlPiUKICBkcGx5cjo6ZmlsdGVyKHBlcmNlbnQubXQgPiBjdXRfcGVyY2VudC5tdCkgJT4lCiAgcm93bmFtZXMoKQoKZmFpbF9wZXJjZW50LnJiID0gc29iakBtZXRhLmRhdGEgJT4lCiAgZHBseXI6OmZpbHRlcihwZXJjZW50LnJiID4gY3V0X3BlcmNlbnQucmIpICU+JQogIHJvd25hbWVzKCkKCmZhaWxfcGVyY2VudC5zdCA9IHNvYmpAbWV0YS5kYXRhICU+JQogIGRwbHlyOjpmaWx0ZXIocGVyY2VudC5zdCA+IGN1dF9wZXJjZW50LnN0KSAlPiUKICByb3duYW1lcygpCgpmYWlsX2xvZzEwX25Db3VudF9STkEgPSBzb2JqQG1ldGEuZGF0YSAlPiUKICBkcGx5cjo6ZmlsdGVyKGxvZzEwX25Db3VudF9STkEgPCBjdXRfbG9nMTBfbkNvdW50X1JOQSkgJT4lCiAgcm93bmFtZXMoKQoKZmFpbF9uRmVhdHVyZV9STkEgPSBzb2JqQG1ldGEuZGF0YSAlPiUKICBkcGx5cjo6ZmlsdGVyKG5GZWF0dXJlX1JOQSA8IGN1dF9uRmVhdHVyZV9STkEpICU+JQogIHJvd25hbWVzKCkKYGBgCgojIyBWaXN1YWxpemF0aW9uCgpUaGlzIGlzIGRpZmZpY3VsdCB0byBoYW5kbGUgdGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGVzZSBtZXRyaWNzIGFjcm9zcyBjZWxscy4gV2Ugb3B0IGZvciB2YXJpb3VzIHZpc3VhbGl6YXRpb24gd2F5czoKCiogKipoaXN0b2dyYW0qKiwgc2hvd2luZyB0aGUgZGlzdHJpYnV0aW9uIG9mIHRoZSBtZXRyaWMKKiAqKnZpb2xpbi9ib3ggcGxvdCoqLCBzaG93aW5nIHRoZSBkaXN0cmlidXRpb24gb2YgdGhlIG1ldHJpYywgdXNlZnVsIGlmIHNldmVyYWwgZGF0YXNldHMgYXJlIGNvbnNpZGVyZWQKKiAqKlVNQVAqKiAob3IgYWx0ZXJuYXRpdmVseSwgdFNORSksIHNob3dpbmcgdGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUgbWV0cmljIG92ZXIgYSAyRCBwcm9qZWN0aW9uIG9mIGNlbGxzCgpZb3UgbWF5IGNob29zZSBvbmUgb2YgdGhlc2UgdmlzdWFsaXphdGlvbiB3YXlzLgoKIyMjIE51bWJlciBvZiBVTUkKCiMjIyMgRGVmaW5lIHRoZSBjb2RlCgpXZSB3cml0ZSBhIGxvdCBvZiBjb2RlIHRvIGNyZWF0ZSBvdXIgZmlndXJlcyBvZiBpbnRlcmVzdDoKCmBgYHtyIGRlZmluZV9jb2RlLCBmaWcud2lkdGggPSAxOCwgZmlnLmhlaWdodCA9IDR9CiMgSGlzdG9ncmFtIHNob3dpbmc6CiMgLSB0aGUgZGlzdHJpYnV0aW9uIG9mIHRoZSBtZXRyaWMKIyAtIGFuIGVzdGltYXRlZCBkZW5zaXR5CiMgLSB0aGUgZmlsdGVyaW5nIHRocmVzaG9sZCBhcyBhIHN0cmFpZ2h0IGxpbmUKcF9oaXN0ID0gZ2dwbG90Mjo6Z2dwbG90KHNvYmpAbWV0YS5kYXRhLCBhZXMoeCA9IGxvZzEwX25Db3VudF9STkEpKSArCiAgZ2dwbG90Mjo6Z2VvbV9oaXN0b2dyYW0oYWVzKHkgPSBnZ3Bsb3QyOjphZnRlcl9zdGF0KGRlbnNpdHkpKSwKICAgICAgICAgICAgICAgICAgICAgICAgICBjb2xvdXIgPSAiYmxhY2siLCBmaWxsID0gIiNGODc2NkQiLCBiaW5zID0gMTAwKSArCiAgZ2dwbG90Mjo6Z2VvbV9kZW5zaXR5KGFscGhhID0gMCwgY29sID0gImJsdWUiLCBsd2QgPSAwLjc1KSArCiAgZ2dwbG90Mjo6Z2VvbV92bGluZSh4aW50ZXJjZXB0ID0gY3V0X2xvZzEwX25Db3VudF9STkEsIGNvbCA9ICJyZWQiKSArCiAgZ2dwbG90Mjo6bGFicyh0aXRsZSA9IHBhc3RlMCgiVGhyZXNob2xkIGZvciBsb2cxMF9uQ291bnRfUk5BIGlzOiAiLCBjdXRfbG9nMTBfbkNvdW50X1JOQSkpICsKICBnZ3Bsb3QyOjp0aGVtZV9jbGFzc2ljKCkgKwogIGdncGxvdDI6OnRoZW1lKHBsb3QudGl0bGUgPSBlbGVtZW50X3RleHQoaGp1c3QgPSAwLjUpKQoKIyBWaW9saW4gcGxvdDoKIyAtIGlzIGVhc2lseSBhY2Nlc3NpYmxlIGluIHRoZSBTZXVyYXQgcGFja2FnZQojIC0gY2FuIG9yIG5vdCBkaXNwbGF5IHRoZSBjZWxscyAoc2V0IGBwdC5zaXplID0gMGAgdG8gaGlkZSB0aGUgY2VsbHMpCiMgLSBpcyB1c2VmdWwgd2hlbiBzZXZlcmFsIGRhdGFzZXRzIGhhdmUgYmVlbiBtZXJnZWQKcF92aW9saW4gPSBTZXVyYXQ6OlZsblBsb3Qoc29iaiwgZmVhdHVyZXMgPSAibG9nMTBfbkNvdW50X1JOQSIpICsKICBnZ3Bsb3QyOjpnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSBjdXRfbG9nMTBfbkNvdW50X1JOQSwgY29sID0gInJlZCIpICsKICBnZ3Bsb3QyOjp0aGVtZShheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCksCiAgICAgICAgICAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiKQoKIyBCb3ggcGxvdCBpcyBzaW1pbGFyIHRvIHZpb2xpbiBwbG90IGJ1dCBub3QgaW4gdGhlIFNldXJhdCBwYWNrYWdlCnBfYm94cGxvdCA9IGdncGxvdDI6OmdncGxvdChzb2JqQG1ldGEuZGF0YSwgYWVzKHkgPSBsb2cxMF9uQ291bnRfUk5BLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB4ID0gb3JpZy5pZGVudCkpICsKICBnZ3Bsb3QyOjpnZW9tX2JveHBsb3QoY29sb3VyID0gImJsYWNrIiwgZmlsbCA9ICIjRjg3NjZEIikgKwogIGdncGxvdDI6Omdlb21faml0dGVyKHdpZHRoID0gMC4zLCBzaXplID0gMC4wMDEpICsKICBnZ3Bsb3QyOjpnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSBjdXRfbG9nMTBfbkNvdW50X1JOQSwgY29sID0gInJlZCIpICsKICBnZ3Bsb3QyOjp0aGVtZV9jbGFzc2ljKCkgKwogIGdncGxvdDI6OnRoZW1lKGF4aXMudGl0bGUueCA9IGVsZW1lbnRfYmxhbmsoKSkKCiMgVGhpcyAyRCByZXByZXNlbnRhdGlvbiBzaG93cyB0aGUgZGlzdHJpYnV0aW9uIG9mIHRoZSBtZXRyaWMgb3ZlciBjZWxscwpwX3VtYXAgPSBTZXVyYXQ6OkZlYXR1cmVQbG90KHNvYmosCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmVkdWN0aW9uID0gInVtYXAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZlYXR1cmVzID0gImxvZzEwX25Db3VudF9STkEiKSArCiAgZ2dwbG90Mjo6c2NhbGVfY29sb3JfZ3JhZGllbnRuKGNvbG9ycyA9IGNvbG9yX3BhbGV0dGUpICsKICBnZ3Bsb3QyOjp0aGVtZShhc3BlY3QucmF0aW8gPSAxKQoKIyBUaGlzIDJEIHJlcHJlc2VudGF0aW9uIHNob3dzIHRoZSBsb3cgcXVhbGl0eSBjZWxscyBpbiBjb2xvcgojIGBvcmRlciA9ICJmYWlsImAgaXMgdXNlZCB0byBkaXNwbGF5IGZhaWxpbmcgY2VsbHMgb24gZnJvbnQKIyBiZWZvcmUsIHdlIG5lZWQgdG8gZGVmaW5lIHRoZSAiZmFpbG9ycGFzcyIgY29sdW1uIGluIHNvYmpAbWV0YS5kYXRhCiMgSXQgY29ycmVzcG9uZHMgdG8gImZhaWwiIGZvciBjZWxscyB0aGF0IGRvIG5vIHBhc3MgdGhlIHRocmVzaG9sZCwKIyBvciAicGFzcyIgb3RoZXJ3aXNlCnNvYmokZmFpbG9ycGFzcyA9IGlmZWxzZShjb2xuYW1lcyhzb2JqKSAlaW4lIGZhaWxfbG9nMTBfbkNvdW50X1JOQSwKICAgICAgICAgICAgICAgICAgICAgICAgIHllcyA9ICJmYWlsIiwgbm8gPSAicGFzcyIpICU+JQogIGFzLmZhY3RvcigpCgpwX2ZhaWwgPSBTZXVyYXQ6OkRpbVBsb3Qoc29iaiwKICAgICAgICAgICAgICAgICAgICAgICAgIGdyb3VwLmJ5ID0gImZhaWxvcnBhc3MiLAogICAgICAgICAgICAgICAgICAgICAgICAgb3JkZXIgPSAiZmFpbCIpICsKICBnZ3Bsb3QyOjpzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYygiI0Y4NzY2RCIsICJncmF5ODAiKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWtzID0gbGV2ZWxzKHNvYmokZmFpbG9ycGFzcykpICsKICBnZ3Bsb3QyOjpsYWJzKHRpdGxlID0gImxvZzEwX25Db3VudF9STkEiLAogICAgICAgICAgICAgICAgc3VidGl0bGUgPSBwYXN0ZTAobGVuZ3RoKGZhaWxfbG9nMTBfbkNvdW50X1JOQSksICIgY2VsbHMgZmFpbCAoIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJvdW5kKDEwMCpsZW5ndGgoZmFpbF9sb2cxMF9uQ291bnRfUk5BKS9uY29sKHNvYmopLCAyKSwgIiAlKSIpKSArCiAgZ2dwbG90Mjo6dGhlbWUoYXNwZWN0LnJhdGlvID0gMSwKICAgICAgICAgICAgICAgICBwbG90LnRpdGxlID0gZWxlbWVudF90ZXh0KGhqdXN0ID0gMC41KSwKICAgICAgICAgICAgICAgICBwbG90LnN1YnRpdGxlID0gZWxlbWVudF90ZXh0KGhqdXN0ID0gMC41KSkKCnNvYmokZmFpbG9ycGFzcyA9IE5VTEwgIyByZW1vdmUgdGhlIGNvbHVtbiAoaXQgd2FzIHRlbXBvcmFyeSkKCiMgV2UgdXNlIHRoZSBwYXRjaHdvcmsgcGFja2FnZSB0byBhcnJhbmdlIGFsbCBmaWd1cmVzIHRvZ2V0aGVyCnBhdGNod29yazo6d3JhcF9wbG90cyhwX3VtYXAsIHBfZmFpbCwgcF9oaXN0LCBwX3Zpb2xpbiwgcF9ib3hwbG90KSArCiAgcGF0Y2h3b3JrOjpwbG90X2xheW91dChucm93ID0gMSwgd2lkdGhzID0gYygxLCAxLCAyLCAxLCAxKSkKYGBgCgojIyMjIERlZmluZSBhIGZ1bmN0aW9uCgpJbnN0ZWFkIG9mIGNvcHlpbmctcGFzdGluZyB0aGlzIHNlY3Rpb24gb2YgY29kZSBmb3IgYWxsIFFDLW1ldHJpY3MsIHdlIGRlc2lnbiBhIGZ1bmN0aW9uIHVzaW5nIHRoZSB0ZW1wbGF0ZSBiZWxvdzoKCmBgYHtyIG15X2Z1bmN0aW9uX25hbWUsIGV2YWwgPSBGQUxTRX0KbXlfZnVuY3Rpb25fbmFtZSA9IGZ1bmN0aW9uKHBhcmFtMSwgcGFyYW0yKSB7CiAgIyBkbyBzb21ldGhpbmcgd2l0aCB0aGUgcGFyYW1ldGVyIHZhbHVlcwogIG91dHB1dCA9ICJzb21ldGhpbmciCiAgCiAgcmV0dXJuKG91dHB1dCkKfQpgYGAKCkhlcmUgaXMgdGhlIGZ1bmN0aW9uOgoKYGBge3IgcWNfcHJpbnRfZnVuY3Rpb259CnByaW50XzFfcWNfbWV0cmljID0gZnVuY3Rpb24ob2JqZWN0ID0gc29iaiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBxYyA9ICJsb2cxMF9uQ291bnRfUk5BIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjdXRfcWMgPSBjdXRfbG9nMTBfbkNvdW50X1JOQSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmYWlsaW5nX2NlbGxzID0gZmFpbF9sb2cxMF9uQ291bnRfUk5BKSB7CiAgIyBEZXNjcmlwdGlvbiBvZiB0aGUgcGFyYW1ldGVyczoKICAjIC0gc29iaiA6IHRoZSBTZXVyYXQgb2JqZWN0LCB3aXRoIGRlZmF1bHQgdmFsdWUgdG8gdGhlIG9uZQogICMgLSBxYyA6IENIQVJBQ1RFUiA6IHRoZSBRQyBtZXRyaWMsIG11c3QgYmUgYSBjb2x1bW4gaW4gc29iakBtZXRhLmRhdGEKICAjIC0gY3V0X3FjIDogTlVNRVJJQyA6IHRoZSBmaWx0ZXJpbmcgdGhyZXNob2xkIGZvciB0aGUgUUMgbWV0cmljCiAgIyAtIGZhaWxpbmdfY2VsbHMgOiBDSEFSQUNURVIgVkVDVE9SIDogdGhlIGNlbGxzIHRoYXQgZmFpbCB0aGUgUUMKICAKICAjIEhpc3RvZ3JhbQogIHBfaGlzdCA9IGdncGxvdDI6OmdncGxvdChvYmplY3RAbWV0YS5kYXRhLCBhZXMoeCA9IC5kYXRhW1txY11dKSkgKwogICAgZ2dwbG90Mjo6Z2VvbV9oaXN0b2dyYW0oYWVzKHkgPSBnZ3Bsb3QyOjphZnRlcl9zdGF0KGRlbnNpdHkpKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbG91ciA9ICJibGFjayIsIGZpbGwgPSAiI0Y4NzY2RCIsIGJpbnMgPSAxMDApICsKICAgIGdncGxvdDI6Omdlb21fZGVuc2l0eShhbHBoYSA9IDAsIGNvbCA9ICJibHVlIiwgbHdkID0gMC43NSkgKwogICAgZ2dwbG90Mjo6Z2VvbV92bGluZSh4aW50ZXJjZXB0ID0gY3V0X3FjLCBjb2wgPSAicmVkIikgKwogICAgZ2dwbG90Mjo6bGFicyh0aXRsZSA9IHBhc3RlMCgiVGhyZXNob2xkIGZvciAiLCBxYywgIiBpczogIiwgY3V0X3FjKSkgKwogICAgZ2dwbG90Mjo6dGhlbWVfY2xhc3NpYygpICsKICAgIGdncGxvdDI6OnRoZW1lKHBsb3QudGl0bGUgPSBlbGVtZW50X3RleHQoaGp1c3QgPSAwLjUpKQogIAogICMgVmlvbGluIHBsb3QKICBwX3Zpb2xpbiA9IFNldXJhdDo6VmxuUGxvdChvYmplY3QsIGZlYXR1cmVzID0gcWMpICsKICAgIGdncGxvdDI6Omdlb21faGxpbmUoeWludGVyY2VwdCA9IGN1dF9xYywgY29sID0gInJlZCIpICsKICAgIGdncGxvdDI6OnRoZW1lKGF4aXMudGl0bGUueCA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICAgICAgICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikKICAKICAjIEJveCBwbG90CiAgcF9ib3hwbG90ID0gZ2dwbG90Mjo6Z2dwbG90KG9iamVjdEBtZXRhLmRhdGEsIGFlcyh5ID0gLmRhdGFbW3FjXV0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB4ID0gIm9yaWcuaWRlbnQiKSkgKwogICAgZ2dwbG90Mjo6Z2VvbV9ib3hwbG90KGNvbG91ciA9ICJibGFjayIsIGZpbGwgPSAiI0Y4NzY2RCIpICsKICAgIGdncGxvdDI6Omdlb21faml0dGVyKHdpZHRoID0gMC4zLCBzaXplID0gMC4wMDEpICsKICAgIGdncGxvdDI6Omdlb21faGxpbmUoeWludGVyY2VwdCA9IGN1dF9xYywgY29sID0gInJlZCIpICsKICAgIGdncGxvdDI6OnRoZW1lX2NsYXNzaWMoKSArCiAgICBnZ3Bsb3QyOjp0aGVtZShheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCkpCiAgCiAgIyBGZWF0dXJlIHBsb3QKICBwX3VtYXAgPSBTZXVyYXQ6OkZlYXR1cmVQbG90KG9iamVjdCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJlZHVjdGlvbiA9ICJ1bWFwIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZlYXR1cmVzID0gcWMpICsKICAgIGdncGxvdDI6OnNjYWxlX2NvbG9yX2dyYWRpZW50bihjb2xvcnMgPSBjb2xvcl9wYWxldHRlKSArCiAgICBnZ3Bsb3QyOjp0aGVtZShhc3BlY3QucmF0aW8gPSAxKQogIAogICMgRGltIHBsb3QKICBvYmplY3QkZmFpbG9ycGFzcyA9IGlmZWxzZShjb2xuYW1lcyhvYmplY3QpICVpbiUgZmFpbGluZ19jZWxscywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICB5ZXMgPSAiZmFpbCIsIG5vID0gInBhc3MiKSAlPiUKICAgIGFzLmZhY3RvcigpCiAgCiAgcF9mYWlsID0gU2V1cmF0OjpEaW1QbG90KG9iamVjdCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgZ3JvdXAuYnkgPSAiZmFpbG9ycGFzcyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgIG9yZGVyID0gImZhaWwiKSArCiAgICBnZ3Bsb3QyOjpzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYygiI0Y4NzY2RCIsICJncmF5ODAiKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBicmVha3MgPSBsZXZlbHMob2JqZWN0JGZhaWxvcnBhc3MpKSArCiAgICBnZ3Bsb3QyOjpsYWJzKHRpdGxlID0gcWMsCiAgICAgICAgICAgICAgICAgIHN1YnRpdGxlID0gcGFzdGUwKGxlbmd0aChmYWlsaW5nX2NlbGxzKSwgIiBjZWxscyBmYWlsICgiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByb3VuZCgxMDAqbGVuZ3RoKGZhaWxpbmdfY2VsbHMpL25jb2woc29iaiksIDIpLCAiICUpIikpICsKICAgIGdncGxvdDI6OnRoZW1lKGFzcGVjdC5yYXRpbyA9IDEsCiAgICAgICAgICAgICAgICAgICBwbG90LnRpdGxlID0gZWxlbWVudF90ZXh0KGhqdXN0ID0gMC41KSwKICAgICAgICAgICAgICAgICAgIHBsb3Quc3VidGl0bGUgPSBlbGVtZW50X3RleHQoaGp1c3QgPSAwLjUpKQogIAogICMgUGF0Y2h3b3JrCiAgcCA9IHBhdGNod29yazo6d3JhcF9wbG90cyhwX3VtYXAsIHBfZmFpbCwgcF9oaXN0LCBwX3Zpb2xpbiwgcF9ib3hwbG90KSArCiAgICBwYXRjaHdvcms6OnBsb3RfbGF5b3V0KG5yb3cgPSAxLCB3aWR0aHMgPSBjKDEsIDEsIDIsIDEsIDEpKQogIAogIHJldHVybihwKQp9CmBgYAoKIyMjIyBDaGVjayB0aGUgZnVuY3Rpb24KClRoaXMgaXMgd29ya2luZyBmb3IgYGxvZzEwX25Db3VudF9STkFgLCBiZWNhdXNlIGl0IGlzIHVzZWQgdG8gZGVmaW5lIGRlZmF1bHQgcGFyYW1ldGVyIHZhbHVlczoKCmBgYHtyIGNoZWNrMV9sb2cxMF9uQ291bnRfUk5BLCBmaWcud2lkdGggPSAxOCwgZmlnLmhlaWdodCA9IDR9CnByaW50XzFfcWNfbWV0cmljKCkKYGBgCgpidXQgYWxzbyB3b3JrcyBieSBzcGVjaWZ5aW5nIHRoZSBwYXJhbWV0ZXIgdmFsdWVzOgoKYGBge3IgY2hlY2syX2xvZzEwX25Db3VudF9STkEsIGZpZy53aWR0aCA9IDE4LCBmaWcuaGVpZ2h0ID0gNH0KcHJpbnRfMV9xY19tZXRyaWMoc29iaiwKICAgICAgICAgICAgICAgICAgcWMgPSAibG9nMTBfbkNvdW50X1JOQSIsCiAgICAgICAgICAgICAgICAgIGN1dF9xYyA9IGN1dF9sb2cxMF9uQ291bnRfUk5BLAogICAgICAgICAgICAgICAgICBmYWlsaW5nX2NlbGxzID0gZmFpbF9sb2cxMF9uQ291bnRfUk5BKQpgYGAKClNvIHdlIGNhbiBjb3B5LXBhc3RlIG9ubHkgdGhpcyBjaHVuayBmb3IgdGhlIG5leHQgUUMgbWV0cmljcyAhCgojIyMgTnVtYmVyIG9mIGdlbmVzCgpgYGB7ciBzZWVfbkZlYXR1cmVfUk5BLCBmaWcud2lkdGggPSAxOCwgZmlnLmhlaWdodCA9IDR9CnByaW50XzFfcWNfbWV0cmljKHNvYmosCiAgICAgICAgICAgICAgICAgIHFjID0gIm5GZWF0dXJlX1JOQSIsCiAgICAgICAgICAgICAgICAgIGN1dF9xYyA9IGN1dF9uRmVhdHVyZV9STkEsCiAgICAgICAgICAgICAgICAgIGZhaWxpbmdfY2VsbHMgPSBmYWlsX25GZWF0dXJlX1JOQSkKYGBgCgojIyMgTWl0b2Nob25kcmlhbCBnZW5lcyBleHByZXNzaW9uCgpgYGB7ciBzZWVfcGVyY2VudC5tdCwgZmlnLndpZHRoID0gMTgsIGZpZy5oZWlnaHQgPSA0fQpwcmludF8xX3FjX21ldHJpYyhzb2JqLAogICAgICAgICAgICAgICAgICBxYyA9ICJwZXJjZW50Lm10IiwKICAgICAgICAgICAgICAgICAgY3V0X3FjID0gY3V0X3BlcmNlbnQubXQsCiAgICAgICAgICAgICAgICAgIGZhaWxpbmdfY2VsbHMgPSBmYWlsX3BlcmNlbnQubXQpCmBgYAoKIyMjIFJpYm9zb21hbCBnZW5lcyBleHByZXNzaW9uCgpgYGB7ciBzZWVfcGVyY2VudC5yYiwgZmlnLndpZHRoID0gMTgsIGZpZy5oZWlnaHQgPSA0fQpwcmludF8xX3FjX21ldHJpYyhzb2JqLAogICAgICAgICAgICAgICAgICBxYyA9ICJwZXJjZW50LnJiIiwKICAgICAgICAgICAgICAgICAgY3V0X3FjID0gY3V0X3BlcmNlbnQucmIsCiAgICAgICAgICAgICAgICAgIGZhaWxpbmdfY2VsbHMgPSBmYWlsX3BlcmNlbnQucmIpCmBgYAoKIyMjIFN0cmVzcy1yZWxhdGVkIGdlbmVzIGV4cHJlc3Npb24KCmBgYHtyIHNlZV9wZXJjZW50LnN0LCBmaWcud2lkdGggPSAxOCwgZmlnLmhlaWdodCA9IDR9CnByaW50XzFfcWNfbWV0cmljKHNvYmosCiAgICAgICAgICAgICAgICAgIHFjID0gInBlcmNlbnQuc3QiLAogICAgICAgICAgICAgICAgICBjdXRfcWMgPSBjdXRfcGVyY2VudC5zdCwKICAgICAgICAgICAgICAgICAgZmFpbGluZ19jZWxscyA9IGZhaWxfcGVyY2VudC5zdCkKYGBgCgojIEZpbHRlcmluZwoKV2UgY291bGQgZmlsdGVyIG91dCBjZWxscyBiYXNlZCBvbiB0aGVzZSA1IFFDIG1ldHJpY3Mgbm93LiBJdCBpcyBhbHNvIHBvc3NpYmxlIHRvIHdhaXQsIHBlcmZvcm0gdmFyaW91cyBhbm5vdGF0aW9ucyBzdWNoIGFzIGNlbGwgdHlwZSBhbm5vdGF0aW9uIG9yIGNlbGwgY3ljbGUgcGhhc2Ugc2NvcmluZywgdG8gYmV0dGVyIGNoYXJhY3Rlcml6ZSB0aGUgbG93IHF1YWxpdHkgY2VsbHMuCgpUbyBmaWx0ZXIgYSBTZXVyYXQgb2JqZWN0LCB3ZSB1c2UgdGhlIGBzdWJzZXRgIGZ1bmN0aW9uOgoKYGBge3IgZmlsdGVyX3NvYmp9CnNvYmpfZmlsdGVyZWQgPSBzdWJzZXQoc29iaiwgaW52ZXJ0ID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgICBjZWxscyA9IHVuaXF1ZShjKGZhaWxfbG9nMTBfbkNvdW50X1JOQSwgZmFpbF9uRmVhdHVyZV9STkEsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmYWlsX3BlcmNlbnQubXQsIGZhaWxfcGVyY2VudC5yYiwgZmFpbF9wZXJjZW50LnN0KSkpCnNvYmpfZmlsdGVyZWQKYGBgCgpXZSBhcmUgZ29pbmcgdG8gc2F2ZSB0aGUgb2JqZWN0IGFubm90YXRlZCBmb3IgY2VsbHMgdGhhdCBmYWlsIHRoZSBxdWFsaXR5IGNvbnRyb2wsIHJlZ2FyZGxlc3MgdGhlIG1ldHJpY3MuIFNvIHdlIGFkZCBhIHNpbmdsZSBjb2x1bW4gdG8gYHNvYmpAbWV0YS5kYXRhYCA6CgpgYGB7ciBhZGRfZmFpbF9xY30Kc29iaiRmYWlsX3FjID0gaWZlbHNlKGNvbG5hbWVzKHNvYmopICVpbiUgYyhmYWlsX2xvZzEwX25Db3VudF9STkEsIGZhaWxfbkZlYXR1cmVfUk5BLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZhaWxfcGVyY2VudC5tdCwgZmFpbF9wZXJjZW50LnJiLCBmYWlsX3BlcmNlbnQuc3QpLAogICAgICAgICAgICAgICAgICAgICAgeWVzID0gImZhaWwiLCBubyA9ICJwYXNzIikKCnRhYmxlKHNvYmokZmFpbF9xYykKYGBgCgoKIyBTYXZlCgpXZSBzYXZlIHRoZSBub24tZmlsdGVyZWQgU2V1cmF0IG9iamVjdDoKCmBgYHtyIHNhdmVfc29ian0Kc2F2ZVJEUyhzb2JqLCBmaWxlID0gcGFzdGUwKG91dGRpciwgIi9zb2JqX1REM0FfcWNfYW5ub3RhdGVkLnJkcyIpKQpgYGAKClRoaXMgU2V1cmF0IG9iamVjdCBjYW4gdGhlbiBiZSB0aGUgaW5wdXQgb2JqZWN0IGZvciBhbm5vdGF0aW9uLCBkZWZpbml0aW9uIG9mIGEgbmV3IHByb2plY3Rpb24gYW5kIGRvd25zdHJlYW0gYW5hbHlzZXMuCgojIFIgc2Vzc2lvbgoKVGhpcyBpcyBhIGdvb2QgcHJhY3RpY2UgdG8gc2hvdyB0aGUgdmVyc2lvbiBvZiB0aGUgcGFja2FnZXMgdXNlZCBpbiB0aGlzIG5vdGVib29rLgoKYGBge3Igc2Vzc2lvbmluZm8sIGNsYXNzLnNvdXJjZSA9ICJmb2xkLWhpZGUifQpzZXNzaW9uSW5mbygpCmBgYAoK