Overview

On Sept 6th, 2017, Gabriel, Sean and Mauro discussed what the biomass functions of the forestr package should look like. The main goal is to incorporate into the default behaviour the new tropical allometries (Chave et al. 2014). We are also interested in allowing more flexibility in the type of input that the user can provide. We plan three functions, to:

  1. decide which formula/equation applies to a given stem: get_allometry()
  2. apply the formula/equation to each stem: biomass_per_individual()
  3. apply biomass_per_individual() within user-defined groups. This is part of the abundance module and won’t be discussed here.

Get allometries, allowing site-, species-, and stem-level input from the user

The forestr package should contain functions to extract wood density from the Global Wood Density Database or CTFS databases. It may contain also allometric equations to estimate stem height. The user should use these functions to have a more complete dataset. The functions in the biomass module would assume that all the available information is already in the main data table, and that the allometric equation will use all the available information.

The function will return allometries only for the “with height” and the “without height” cases. It will return NA’s if there is no DBH or wood density. This mirrors the available defaults for tropical forests in Chave et al. (2014), Eq. 4 and 7.

The allometric equation to use may potentially depend on the site (or conditions of the site, E), the species (e.g. different equations for palms), and the stem (e.g. different equations for broken trees). The user should be allowed to provide input at the three levels.

The general flow in the get_allometry() function is:

  1. generate default allometries.
  2. overwrite with user-provided site-level allometries.
  3. overwrite with user-provided species-level allometries.
  4. overwrite with user-provided individual-level allometries.

The site-level input could take these forms:

  1. NULL: do nothing.
  2. a vector or a list with one or two allometric equations (for the with/without height cases)

The species-level input could take these forms:

  1. NULL: do nothing.
  2. a list, named with species codes, with one or two allometric equations per species.

If the user has family-level allometries, e.g. for palms, it should not be difficult to translate that into a species-level input. In any case, those could be accessory functions to be developed elsewhere within the package.

The stem-level input could take these forms:

  1. NULL: do nothing.
  2. a list, named with StemID’s, with one or two allometric equations per stem.

The function returns one and only one allometry per stem, based on data availability. It also returns what is the level of estimation for the selected allometry. It returns a table with as many rows as rows has the main dataset. If we find that performance is an issue, we would get rid of the stem-level options, and the function will work on (and return) a table with as many rows as species has the main dataset.

Note that the functions assume that stem height is called “H”, DBH is called “D” and wood density (wood specific gravity) is called “WSG”. This applies both to the allometric equations and the column names in the dataset. The get_allometry() function in particular relies on grepl("H", ...).

get_allometry <- function(x, siteID = NULL, E = NULL, site_level_allometry = NULL, species_level_allometry = NULL, stem_level_allometry = NULL)
{
  # Determine the E value
  if(is.null(E))
  {
    if(is.null(siteID)) siteID = attr(x, "siteID")
    E = CTFS.E[siteID] # this will return zero length if unknown site
    if(length(E) == 0) E = NA
  }
  
  # Generate defaults using Chave et al. 2014. When height data are available,
  # a pantropical model is the best default (Eq. 4). When height data is not available,
  # then local models based on the E climatic parameter are the best default (Eq. 7).
  COLNAMES <- c("with.H", "without.H", "with.H.level", "without.H.level")
  allometries <- matrix(NA, nrow = nrow(x), ncol = length(COLNAMES), dimnames = list(NULL, COLNAMES))
  allometries[, "with.H"] <- paste0("0.0673 * (WSG * D^2 * H)^0.976")
  if(!is.na(E)) allometries[, "without.H"] <- paste0("exp(-1.803 - 0.976 * ", E, " + 0.976 * log(WSG) + 2.673 * log(D) - 0.0299 *(log(D)^2))")
  allometries[, "with.H.level"] <- "Chave et al. 2014, Eq. 4"
  if(!is.na(E)) allometries[, "without.H.level"] <- "Chave et al. 2014, Eq. 7"

  # Incorporate site-level input
  if(!is.null(site_level_allometry))
  {
    site_level_allometry <- unique(unlist(site_level_allometry))
    with.H <- site_level_allometry[grepl("H", site_level_allometry)]
    without.H <- site_level_allometry[!grepl("H", site_level_allometry)]
    
    if(length(with.H) == 1)
    {
      allometries[, "with.H"] <- with.H
      allometries[, "with.H.level"] <- "user-provided site-level allometry, using height"
    }
    
    if(length(without.H) == 1)
    {
      allometries[, "without.H"] <- without.H
      allometries[, "without.H.level"] <- "user-provided site-level allometry, without using height"
    }
    
  }
  
  # Incorporate species-level input
  #species_level_allometry <- list(a = c("2 * D * H", "3 * D"), b = c("2 * D"))
  if(!is.null(species_level_allometry))
  {
    with.H <- sapply(species_level_allometry, function(a) {
      a <- unique(a[grepl("H", a)])
      if(length(a) != 1) a = NA
      a
    })
    
    without.H <- sapply(species_level_allometry, function(a) {
      a <- unique(a[!grepl("H", a)])
      if(length(a) != 1) a = NA
      a
    })
    
    # Overwrite in the table:
    spp <- x$sp
    #spp <- c("a", "b", "c", "a")
    allometries[!is.na(with.H[spp]), "with.H"] <- with.H[spp][!is.na(with.H[spp])]
    allometries[!is.na(with.H[spp]), "with.H.level"] <- "user-provided species-level allometry, using height"
    allometries[!is.na(without.H[spp]), "without.H"] <- without.H[spp][!is.na(without.H[spp])]
    allometries[!is.na(without.H[spp]), "without.H.level"] <- "user-provided species-level allometry, without using height"
  }
  
  # Incorporate stem-level input
  #stem_level_allometry <- list(a = c("2 * D * H", "3 * D"), b = c("2 * D"))
  if(!is.null(stem_level_allometry))
  {
    with.H <- sapply(stem_level_allometry, function(a) {
      a <- unique(a[grepl("H", a)])
      if(length(a) != 1) a = NA
      a
    })
    
    without.H <- sapply(stem_level_allometry, function(a) {
      a <- unique(a[!grepl("H", a)])
      if(length(a) != 1) a = NA
      a
    })
    
    # Overwrite in the table:
    id <- x$StemID
    #spp <- c("a", "b", "c", "a")
    allometries[!is.na(with.H[spp]), "with.H"] <- with.H[spp][!is.na(with.H[spp])]
    allometries[!is.na(with.H[spp]), "with.H.level"] <- "user-provided stem-level allometry, using height"
    allometries[!is.na(without.H[spp]), "without.H"] <- without.H[spp][!is.na(without.H[spp])]
    allometries[!is.na(without.H[spp]), "without.H.level"] <- "user-provided stem-level allometry, without using height"
  }
  
  # Decide which allometry to use:
  allometry <- matrix(NA, nrow = nrow(x), ncol = 2, dimnames = list(x$StemID, c("allometry", "level")))
  with.H <- which(!is.na(x$D) & !is.na(x$WSG) & !is.na(x$H))
  without.H <- which(!is.na(x$D) & !is.na(x$WSG) & is.na(x$H))
  allometry[with.H, "allometry"] <- allometries[with.H, "with.H"]
  allometry[with.H, "level"] <- allometries[with.H, "with.H.level"]
  allometry[without.H, "allometry"] <- allometries[without.H, "without.H"]
  allometry[without.H, "level"] <- allometries[without.H, "without.H.level"]
  return(allometry)
}

Applying allometric equations to stems

The biomass_per_individual() function would apply the equations in a vectorized way for each chunk of stems sharing the same allometry. The allometry and the level of estimation is part of the output of this function, as it is potentially useful metadata (e.g. to report methods in publications).

The allometries argument is the output from the get_allometry() function. Since allometries are stored as raw text, a simple combination of parse and eval should work.


biomass_per_individual <- function(x, allometries)
{
    # then do this:
    allometries.id <- unique(allometries[, "allometry"])
    agb <- rep(NA, nrow(x))
    for(a in allometries.id)
    {
         i <- which(allometries[, "allometry"] == a)
         agb[i] <- eval(parse(text = a), envir = x[i,])
    }
    attr(agb, "allometries") <- allometries
    return(agb)
}

Example

library(bciex)

x <- bci12s1mini
x <- x[!is.na(x$dbh),]
x$dbh[c(3, 5, 7, 9)] <- NA
x$StemID <- paste0(x$treeID, "-", x$stemID)
x$D <- x$dbh
x$H <- NA
x$H[2:5] <- 2:5 * 10
x$WSG <- runif(nrow(x), 0.3, 1)

CTFS.E <- c(bci = 0.5, pasoh = 0.7, xxxx = 1.5)

a <- get_allometry(x = x, siteID = NULL, E = NULL, site_level_allometry = NULL, species_level_allometry = NULL, stem_level_allometry = NULL)
head(a, 10)

a <- get_allometry(x = x, siteID = "bci", E = NULL, site_level_allometry = NULL, species_level_allometry = NULL, stem_level_allometry = NULL)
head(a, 10)

a <- get_allometry(x = x, siteID = NULL, E = 777, site_level_allometry = NULL, species_level_allometry = NULL, stem_level_allometry = NULL)
head(a, 10)

s <- c("2 + D")
a <- get_allometry(x = x, siteID = "bci", E = NULL, site_level_allometry = s, species_level_allometry = NULL, stem_level_allometry = NULL)
head(a, 10)

s1 <- c("2 + D")
s2 <- list(c("2 + D", "2 * D + H"), c("3 * D"))
a <- get_allometry(x = x, siteID = "bci", E = NULL, site_level_allometry = s1, species_level_allometry = s2, stem_level_allometry = NULL)
head(a, 10)

s1 <- c("2 + D")
s2 <- list(apeime = c("2 + D", "2 * D + H"), quaras = c("3 * D"))
a <- get_allometry(x = x, siteID = "bci", E = NULL, site_level_allometry = s1, species_level_allometry = s2, stem_level_allometry = NULL)
head(a, 10)


# This is the most typical case:
a <- get_allometry(x = x, siteID = "bci", E = NULL, site_level_allometry = NULL, species_level_allometry = NULL, stem_level_allometry = NULL)
head(a, 10)

agb.per.i <- biomass_per_individual(x, allometries = a)
head(agb.per.i, 10)
LS0tDQp0aXRsZTogIkJpb21hc3MgbW9kdWxlIGluIGZvcmVzdHIiDQphdXRob3I6ICJHYWJyaWVsIEFyZWxsYW5vLCBTZWFuIE1jTWFob24gJiBNYXVybyBMZXBvcmUiDQpkYXRlOiAnMjAxNy0wOS0yMScNCm91dHB1dDogaHRtbF9ub3RlYm9vaw0KLS0tDQoNCmBgYHtyIHNldHVwLCBlY2hvID0gRkFMU0UsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9DQojIGhhZGxleSdzIHNldHRpbmdzDQpzZXQuc2VlZCgxMDE0KQ0Kb3B0aW9ucyhkaWdpdHMgPSAzKQ0KDQprbml0cjo6b3B0c19jaHVuayRzZXQoDQogIGVjaG8gPSBUUlVFLCAgIyB7bWluZX0NCiAgY29tbWVudCA9ICIjPiIsDQogIGNvbGxhcHNlID0gVFJVRSwNCiAgIyBjYWNoZSA9IFRSVUUsDQogIG91dC53aWR0aCA9ICI3MCUiLA0KICBmaWcuYWxpZ24gPSAiY2VudGVyIiwNCiAgZmlnLndpZHRoID0gNiwNCiAgZmlnLmFzcCA9IDAuNjE4LCAgIyAxIC8gcGhpDQogIGZpZy5zaG93ID0gImhvbGQiDQopDQoNCm9wdGlvbnMoZHBseXIucHJpbnRfbWluID0gNiwgZHBseXIucHJpbnRfbWF4ID0gNikNCmBgYA0KDQoNCiMjIE92ZXJ2aWV3DQoNCk9uIFNlcHQgNnRoLCAyMDE3LCBHYWJyaWVsLCBTZWFuIGFuZCBNYXVybyBkaXNjdXNzZWQgd2hhdCB0aGUgYmlvbWFzcyBmdW5jdGlvbnMgb2YgdGhlIF9fZm9yZXN0cl9fIHBhY2thZ2Ugc2hvdWxkIGxvb2sgbGlrZS4gVGhlIG1haW4gZ29hbCBpcyB0byBpbmNvcnBvcmF0ZSBpbnRvIHRoZSBkZWZhdWx0IGJlaGF2aW91ciB0aGUgbmV3IHRyb3BpY2FsIGFsbG9tZXRyaWVzIChDaGF2ZSBldCBhbC4gMjAxNCkuIFdlIGFyZSBhbHNvIGludGVyZXN0ZWQgaW4gYWxsb3dpbmcgbW9yZSBmbGV4aWJpbGl0eSBpbiB0aGUgdHlwZSBvZiBpbnB1dCB0aGF0IHRoZSB1c2VyIGNhbiBwcm92aWRlLiBXZSBwbGFuIHRocmVlIGZ1bmN0aW9ucywgdG86DQoNCjEpIGRlY2lkZSB3aGljaCBmb3JtdWxhL2VxdWF0aW9uIGFwcGxpZXMgdG8gYSBnaXZlbiBzdGVtOiBgZ2V0X2FsbG9tZXRyeSgpYA0KMikgYXBwbHkgdGhlIGZvcm11bGEvZXF1YXRpb24gdG8gZWFjaCBzdGVtOiBgYmlvbWFzc19wZXJfaW5kaXZpZHVhbCgpYA0KMykgYXBwbHkgYGJpb21hc3NfcGVyX2luZGl2aWR1YWwoKWAgd2l0aGluIHVzZXItZGVmaW5lZCBncm91cHMuIFRoaXMgaXMgcGFydCBvZiB0aGUgYGFidW5kYW5jZWAgbW9kdWxlIGFuZCB3b24ndCBiZSBkaXNjdXNzZWQgaGVyZS4NCg0KIyMgR2V0IGFsbG9tZXRyaWVzLCBhbGxvd2luZyBzaXRlLSwgc3BlY2llcy0sIGFuZCBzdGVtLWxldmVsIGlucHV0IGZyb20gdGhlIHVzZXINCg0KVGhlIF9fZm9yZXN0cl9fIHBhY2thZ2Ugc2hvdWxkIGNvbnRhaW4gZnVuY3Rpb25zIHRvIGV4dHJhY3Qgd29vZCBkZW5zaXR5IGZyb20gdGhlIEdsb2JhbCBXb29kIERlbnNpdHkgRGF0YWJhc2Ugb3IgQ1RGUyBkYXRhYmFzZXMuIEl0IG1heSBjb250YWluIGFsc28gYWxsb21ldHJpYyBlcXVhdGlvbnMgdG8gZXN0aW1hdGUgc3RlbSBoZWlnaHQuIFRoZSB1c2VyIHNob3VsZCB1c2UgdGhlc2UgZnVuY3Rpb25zIHRvIGhhdmUgYSBtb3JlIGNvbXBsZXRlIGRhdGFzZXQuIFRoZSBmdW5jdGlvbnMgaW4gdGhlIGBiaW9tYXNzYCBtb2R1bGUgd291bGQgYXNzdW1lIHRoYXQgYWxsIHRoZSBhdmFpbGFibGUgaW5mb3JtYXRpb24gaXMgYWxyZWFkeSBpbiB0aGUgbWFpbiBkYXRhIHRhYmxlLCBhbmQgdGhhdCB0aGUgYWxsb21ldHJpYyBlcXVhdGlvbiB3aWxsIHVzZSBhbGwgdGhlIGF2YWlsYWJsZSBpbmZvcm1hdGlvbi4NCg0KVGhlIGZ1bmN0aW9uIHdpbGwgcmV0dXJuIGFsbG9tZXRyaWVzIG9ubHkgZm9yIHRoZSAid2l0aCBoZWlnaHQiIGFuZCB0aGUgIndpdGhvdXQgaGVpZ2h0IiBjYXNlcy4gSXQgd2lsbCByZXR1cm4gTkEncyBpZiB0aGVyZSBpcyBubyBEQkggb3Igd29vZCBkZW5zaXR5LiBUaGlzIG1pcnJvcnMgdGhlIGF2YWlsYWJsZSBkZWZhdWx0cyBmb3IgdHJvcGljYWwgZm9yZXN0cyBpbiBDaGF2ZSBldCBhbC4gKDIwMTQpLCBFcS4gNCBhbmQgNy4NCg0KVGhlIGFsbG9tZXRyaWMgZXF1YXRpb24gdG8gdXNlIG1heSBwb3RlbnRpYWxseSBkZXBlbmQgb24gdGhlIHNpdGUgKG9yIGNvbmRpdGlvbnMgb2YgdGhlIHNpdGUsIEUpLCB0aGUgc3BlY2llcyAoZS5nLiBkaWZmZXJlbnQgZXF1YXRpb25zIGZvciBwYWxtcyksIGFuZCB0aGUgc3RlbSAoZS5nLiBkaWZmZXJlbnQgZXF1YXRpb25zIGZvciBicm9rZW4gdHJlZXMpLiBUaGUgdXNlciBzaG91bGQgYmUgYWxsb3dlZCB0byBwcm92aWRlIGlucHV0IGF0IHRoZSB0aHJlZSBsZXZlbHMuDQoNClRoZSBnZW5lcmFsIGZsb3cgaW4gdGhlIGBnZXRfYWxsb21ldHJ5KClgIGZ1bmN0aW9uIGlzOg0KDQoxKSBnZW5lcmF0ZSBkZWZhdWx0IGFsbG9tZXRyaWVzLg0KMikgb3ZlcndyaXRlIHdpdGggdXNlci1wcm92aWRlZCBzaXRlLWxldmVsIGFsbG9tZXRyaWVzLg0KMykgb3ZlcndyaXRlIHdpdGggdXNlci1wcm92aWRlZCBzcGVjaWVzLWxldmVsIGFsbG9tZXRyaWVzLg0KNCkgb3ZlcndyaXRlIHdpdGggdXNlci1wcm92aWRlZCBpbmRpdmlkdWFsLWxldmVsIGFsbG9tZXRyaWVzLg0KDQpUaGUgc2l0ZS1sZXZlbCBpbnB1dCBjb3VsZCB0YWtlIHRoZXNlIGZvcm1zOg0KDQoxKSBOVUxMOiBkbyBub3RoaW5nLg0KMikgYSB2ZWN0b3Igb3IgYSBsaXN0IHdpdGggb25lIG9yIHR3byBhbGxvbWV0cmljIGVxdWF0aW9ucyAoZm9yIHRoZSB3aXRoL3dpdGhvdXQgaGVpZ2h0IGNhc2VzKQ0KDQpUaGUgc3BlY2llcy1sZXZlbCBpbnB1dCBjb3VsZCB0YWtlIHRoZXNlIGZvcm1zOg0KDQoxKSBOVUxMOiBkbyBub3RoaW5nLg0KMikgYSBsaXN0LCBuYW1lZCB3aXRoIHNwZWNpZXMgY29kZXMsIHdpdGggb25lIG9yIHR3byBhbGxvbWV0cmljIGVxdWF0aW9ucyBwZXIgc3BlY2llcy4NCg0KSWYgdGhlIHVzZXIgaGFzIGZhbWlseS1sZXZlbCBhbGxvbWV0cmllcywgZS5nLiBmb3IgcGFsbXMsIGl0IHNob3VsZCBub3QgYmUgZGlmZmljdWx0IHRvIHRyYW5zbGF0ZSB0aGF0IGludG8gYSBzcGVjaWVzLWxldmVsIGlucHV0LiBJbiBhbnkgY2FzZSwgdGhvc2UgY291bGQgYmUgYWNjZXNzb3J5IGZ1bmN0aW9ucyB0byBiZSBkZXZlbG9wZWQgZWxzZXdoZXJlIHdpdGhpbiB0aGUgcGFja2FnZS4NCg0KVGhlIHN0ZW0tbGV2ZWwgaW5wdXQgY291bGQgdGFrZSB0aGVzZSBmb3JtczoNCg0KMSkgTlVMTDogZG8gbm90aGluZy4NCjIpIGEgbGlzdCwgbmFtZWQgd2l0aCBTdGVtSUQncywgd2l0aCBvbmUgb3IgdHdvIGFsbG9tZXRyaWMgZXF1YXRpb25zIHBlciBzdGVtLg0KDQpUaGUgZnVuY3Rpb24gcmV0dXJucyBvbmUgYW5kIG9ubHkgb25lIGFsbG9tZXRyeSBwZXIgc3RlbSwgYmFzZWQgb24gZGF0YSBhdmFpbGFiaWxpdHkuIEl0IGFsc28gcmV0dXJucyB3aGF0IGlzIHRoZSBsZXZlbCBvZiBlc3RpbWF0aW9uIGZvciB0aGUgc2VsZWN0ZWQgYWxsb21ldHJ5LiBJdCByZXR1cm5zIGEgdGFibGUgd2l0aCBhcyBtYW55IHJvd3MgYXMgcm93cyBoYXMgdGhlIG1haW4gZGF0YXNldC4gSWYgd2UgZmluZCB0aGF0IHBlcmZvcm1hbmNlIGlzIGFuIGlzc3VlLCB3ZSB3b3VsZCBnZXQgcmlkIG9mIHRoZSBzdGVtLWxldmVsIG9wdGlvbnMsIGFuZCB0aGUgZnVuY3Rpb24gd2lsbCB3b3JrIG9uIChhbmQgcmV0dXJuKSBhIHRhYmxlIHdpdGggYXMgbWFueSByb3dzIGFzIHNwZWNpZXMgaGFzIHRoZSBtYWluIGRhdGFzZXQuDQoNCk5vdGUgdGhhdCB0aGUgZnVuY3Rpb25zIGFzc3VtZSB0aGF0IHN0ZW0gaGVpZ2h0IGlzIGNhbGxlZCAiSCIsIERCSCBpcyBjYWxsZWQgIkQiIGFuZCB3b29kIGRlbnNpdHkgKHdvb2Qgc3BlY2lmaWMgZ3Jhdml0eSkgaXMgY2FsbGVkICJXU0ciLiBUaGlzIGFwcGxpZXMgYm90aCB0byB0aGUgYWxsb21ldHJpYyBlcXVhdGlvbnMgYW5kIHRoZSBjb2x1bW4gbmFtZXMgaW4gdGhlIGRhdGFzZXQuIFRoZSBgZ2V0X2FsbG9tZXRyeSgpYCBmdW5jdGlvbiBpbiBwYXJ0aWN1bGFyIHJlbGllcyBvbiBgZ3JlcGwoIkgiLCAuLi4pYC4NCg0KYGBge3J9DQpnZXRfYWxsb21ldHJ5IDwtIGZ1bmN0aW9uKHgsIHNpdGVJRCA9IE5VTEwsIEUgPSBOVUxMLCBzaXRlX2xldmVsX2FsbG9tZXRyeSA9IE5VTEwsIHNwZWNpZXNfbGV2ZWxfYWxsb21ldHJ5ID0gTlVMTCwgc3RlbV9sZXZlbF9hbGxvbWV0cnkgPSBOVUxMKQ0Kew0KICAjIERldGVybWluZSB0aGUgRSB2YWx1ZQ0KICBpZihpcy5udWxsKEUpKQ0KICB7DQogICAgaWYoaXMubnVsbChzaXRlSUQpKSBzaXRlSUQgPSBhdHRyKHgsICJzaXRlSUQiKQ0KICAgIEUgPSBDVEZTLkVbc2l0ZUlEXSAjIHRoaXMgd2lsbCByZXR1cm4gemVybyBsZW5ndGggaWYgdW5rbm93biBzaXRlDQogICAgaWYobGVuZ3RoKEUpID09IDApIEUgPSBOQQ0KICB9DQogIA0KICAjIEdlbmVyYXRlIGRlZmF1bHRzIHVzaW5nIENoYXZlIGV0IGFsLiAyMDE0LiBXaGVuIGhlaWdodCBkYXRhIGFyZSBhdmFpbGFibGUsDQogICMgYSBwYW50cm9waWNhbCBtb2RlbCBpcyB0aGUgYmVzdCBkZWZhdWx0IChFcS4gNCkuIFdoZW4gaGVpZ2h0IGRhdGEgaXMgbm90IGF2YWlsYWJsZSwNCiAgIyB0aGVuIGxvY2FsIG1vZGVscyBiYXNlZCBvbiB0aGUgRSBjbGltYXRpYyBwYXJhbWV0ZXIgYXJlIHRoZSBiZXN0IGRlZmF1bHQgKEVxLiA3KS4NCiAgQ09MTkFNRVMgPC0gYygid2l0aC5IIiwgIndpdGhvdXQuSCIsICJ3aXRoLkgubGV2ZWwiLCAid2l0aG91dC5ILmxldmVsIikNCiAgYWxsb21ldHJpZXMgPC0gbWF0cml4KE5BLCBucm93ID0gbnJvdyh4KSwgbmNvbCA9IGxlbmd0aChDT0xOQU1FUyksIGRpbW5hbWVzID0gbGlzdChOVUxMLCBDT0xOQU1FUykpDQogIGFsbG9tZXRyaWVzWywgIndpdGguSCJdIDwtIHBhc3RlMCgiMC4wNjczICogKFdTRyAqIEReMiAqIEgpXjAuOTc2IikNCiAgaWYoIWlzLm5hKEUpKSBhbGxvbWV0cmllc1ssICJ3aXRob3V0LkgiXSA8LSBwYXN0ZTAoImV4cCgtMS44MDMgLSAwLjk3NiAqICIsIEUsICIgKyAwLjk3NiAqIGxvZyhXU0cpICsgMi42NzMgKiBsb2coRCkgLSAwLjAyOTkgKihsb2coRCleMikpIikNCiAgYWxsb21ldHJpZXNbLCAid2l0aC5ILmxldmVsIl0gPC0gIkNoYXZlIGV0IGFsLiAyMDE0LCBFcS4gNCINCiAgaWYoIWlzLm5hKEUpKSBhbGxvbWV0cmllc1ssICJ3aXRob3V0LkgubGV2ZWwiXSA8LSAiQ2hhdmUgZXQgYWwuIDIwMTQsIEVxLiA3Ig0KDQogICMgSW5jb3Jwb3JhdGUgc2l0ZS1sZXZlbCBpbnB1dA0KICBpZighaXMubnVsbChzaXRlX2xldmVsX2FsbG9tZXRyeSkpDQogIHsNCiAgICBzaXRlX2xldmVsX2FsbG9tZXRyeSA8LSB1bmlxdWUodW5saXN0KHNpdGVfbGV2ZWxfYWxsb21ldHJ5KSkNCiAgICB3aXRoLkggPC0gc2l0ZV9sZXZlbF9hbGxvbWV0cnlbZ3JlcGwoIkgiLCBzaXRlX2xldmVsX2FsbG9tZXRyeSldDQogICAgd2l0aG91dC5IIDwtIHNpdGVfbGV2ZWxfYWxsb21ldHJ5WyFncmVwbCgiSCIsIHNpdGVfbGV2ZWxfYWxsb21ldHJ5KV0NCiAgICANCiAgICBpZihsZW5ndGgod2l0aC5IKSA9PSAxKQ0KICAgIHsNCiAgICAgIGFsbG9tZXRyaWVzWywgIndpdGguSCJdIDwtIHdpdGguSA0KICAgICAgYWxsb21ldHJpZXNbLCAid2l0aC5ILmxldmVsIl0gPC0gInVzZXItcHJvdmlkZWQgc2l0ZS1sZXZlbCBhbGxvbWV0cnksIHVzaW5nIGhlaWdodCINCiAgICB9DQogICAgDQogICAgaWYobGVuZ3RoKHdpdGhvdXQuSCkgPT0gMSkNCiAgICB7DQogICAgICBhbGxvbWV0cmllc1ssICJ3aXRob3V0LkgiXSA8LSB3aXRob3V0LkgNCiAgICAgIGFsbG9tZXRyaWVzWywgIndpdGhvdXQuSC5sZXZlbCJdIDwtICJ1c2VyLXByb3ZpZGVkIHNpdGUtbGV2ZWwgYWxsb21ldHJ5LCB3aXRob3V0IHVzaW5nIGhlaWdodCINCiAgICB9DQogICAgDQogIH0NCiAgDQogICMgSW5jb3Jwb3JhdGUgc3BlY2llcy1sZXZlbCBpbnB1dA0KICAjc3BlY2llc19sZXZlbF9hbGxvbWV0cnkgPC0gbGlzdChhID0gYygiMiAqIEQgKiBIIiwgIjMgKiBEIiksIGIgPSBjKCIyICogRCIpKQ0KICBpZighaXMubnVsbChzcGVjaWVzX2xldmVsX2FsbG9tZXRyeSkpDQogIHsNCiAgICB3aXRoLkggPC0gc2FwcGx5KHNwZWNpZXNfbGV2ZWxfYWxsb21ldHJ5LCBmdW5jdGlvbihhKSB7DQogICAgICBhIDwtIHVuaXF1ZShhW2dyZXBsKCJIIiwgYSldKQ0KICAgICAgaWYobGVuZ3RoKGEpICE9IDEpIGEgPSBOQQ0KICAgICAgYQ0KICAgIH0pDQogICAgDQogICAgd2l0aG91dC5IIDwtIHNhcHBseShzcGVjaWVzX2xldmVsX2FsbG9tZXRyeSwgZnVuY3Rpb24oYSkgew0KICAgICAgYSA8LSB1bmlxdWUoYVshZ3JlcGwoIkgiLCBhKV0pDQogICAgICBpZihsZW5ndGgoYSkgIT0gMSkgYSA9IE5BDQogICAgICBhDQogICAgfSkNCiAgICANCiAgICAjIE92ZXJ3cml0ZSBpbiB0aGUgdGFibGU6DQogICAgc3BwIDwtIHgkc3ANCiAgICAjc3BwIDwtIGMoImEiLCAiYiIsICJjIiwgImEiKQ0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRoLkhbc3BwXSksICJ3aXRoLkgiXSA8LSB3aXRoLkhbc3BwXVshaXMubmEod2l0aC5IW3NwcF0pXQ0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRoLkhbc3BwXSksICJ3aXRoLkgubGV2ZWwiXSA8LSAidXNlci1wcm92aWRlZCBzcGVjaWVzLWxldmVsIGFsbG9tZXRyeSwgdXNpbmcgaGVpZ2h0Ig0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRob3V0Lkhbc3BwXSksICJ3aXRob3V0LkgiXSA8LSB3aXRob3V0Lkhbc3BwXVshaXMubmEod2l0aG91dC5IW3NwcF0pXQ0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRob3V0Lkhbc3BwXSksICJ3aXRob3V0LkgubGV2ZWwiXSA8LSAidXNlci1wcm92aWRlZCBzcGVjaWVzLWxldmVsIGFsbG9tZXRyeSwgd2l0aG91dCB1c2luZyBoZWlnaHQiDQogIH0NCiAgDQogICMgSW5jb3Jwb3JhdGUgc3RlbS1sZXZlbCBpbnB1dA0KICAjc3RlbV9sZXZlbF9hbGxvbWV0cnkgPC0gbGlzdChhID0gYygiMiAqIEQgKiBIIiwgIjMgKiBEIiksIGIgPSBjKCIyICogRCIpKQ0KICBpZighaXMubnVsbChzdGVtX2xldmVsX2FsbG9tZXRyeSkpDQogIHsNCiAgICB3aXRoLkggPC0gc2FwcGx5KHN0ZW1fbGV2ZWxfYWxsb21ldHJ5LCBmdW5jdGlvbihhKSB7DQogICAgICBhIDwtIHVuaXF1ZShhW2dyZXBsKCJIIiwgYSldKQ0KICAgICAgaWYobGVuZ3RoKGEpICE9IDEpIGEgPSBOQQ0KICAgICAgYQ0KICAgIH0pDQogICAgDQogICAgd2l0aG91dC5IIDwtIHNhcHBseShzdGVtX2xldmVsX2FsbG9tZXRyeSwgZnVuY3Rpb24oYSkgew0KICAgICAgYSA8LSB1bmlxdWUoYVshZ3JlcGwoIkgiLCBhKV0pDQogICAgICBpZihsZW5ndGgoYSkgIT0gMSkgYSA9IE5BDQogICAgICBhDQogICAgfSkNCiAgICANCiAgICAjIE92ZXJ3cml0ZSBpbiB0aGUgdGFibGU6DQogICAgaWQgPC0geCRTdGVtSUQNCiAgICAjc3BwIDwtIGMoImEiLCAiYiIsICJjIiwgImEiKQ0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRoLkhbc3BwXSksICJ3aXRoLkgiXSA8LSB3aXRoLkhbc3BwXVshaXMubmEod2l0aC5IW3NwcF0pXQ0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRoLkhbc3BwXSksICJ3aXRoLkgubGV2ZWwiXSA8LSAidXNlci1wcm92aWRlZCBzdGVtLWxldmVsIGFsbG9tZXRyeSwgdXNpbmcgaGVpZ2h0Ig0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRob3V0Lkhbc3BwXSksICJ3aXRob3V0LkgiXSA8LSB3aXRob3V0Lkhbc3BwXVshaXMubmEod2l0aG91dC5IW3NwcF0pXQ0KICAgIGFsbG9tZXRyaWVzWyFpcy5uYSh3aXRob3V0Lkhbc3BwXSksICJ3aXRob3V0LkgubGV2ZWwiXSA8LSAidXNlci1wcm92aWRlZCBzdGVtLWxldmVsIGFsbG9tZXRyeSwgd2l0aG91dCB1c2luZyBoZWlnaHQiDQogIH0NCiAgDQogICMgRGVjaWRlIHdoaWNoIGFsbG9tZXRyeSB0byB1c2U6DQogIGFsbG9tZXRyeSA8LSBtYXRyaXgoTkEsIG5yb3cgPSBucm93KHgpLCBuY29sID0gMiwgZGltbmFtZXMgPSBsaXN0KHgkU3RlbUlELCBjKCJhbGxvbWV0cnkiLCAibGV2ZWwiKSkpDQogIHdpdGguSCA8LSB3aGljaCghaXMubmEoeCREKSAmICFpcy5uYSh4JFdTRykgJiAhaXMubmEoeCRIKSkNCiAgd2l0aG91dC5IIDwtIHdoaWNoKCFpcy5uYSh4JEQpICYgIWlzLm5hKHgkV1NHKSAmIGlzLm5hKHgkSCkpDQogIGFsbG9tZXRyeVt3aXRoLkgsICJhbGxvbWV0cnkiXSA8LSBhbGxvbWV0cmllc1t3aXRoLkgsICJ3aXRoLkgiXQ0KICBhbGxvbWV0cnlbd2l0aC5ILCAibGV2ZWwiXSA8LSBhbGxvbWV0cmllc1t3aXRoLkgsICJ3aXRoLkgubGV2ZWwiXQ0KICBhbGxvbWV0cnlbd2l0aG91dC5ILCAiYWxsb21ldHJ5Il0gPC0gYWxsb21ldHJpZXNbd2l0aG91dC5ILCAid2l0aG91dC5IIl0NCiAgYWxsb21ldHJ5W3dpdGhvdXQuSCwgImxldmVsIl0gPC0gYWxsb21ldHJpZXNbd2l0aG91dC5ILCAid2l0aG91dC5ILmxldmVsIl0NCiAgcmV0dXJuKGFsbG9tZXRyeSkNCn0NCg0KYGBgDQoNCiMjIEFwcGx5aW5nIGFsbG9tZXRyaWMgZXF1YXRpb25zIHRvIHN0ZW1zDQoNClRoZSBgYmlvbWFzc19wZXJfaW5kaXZpZHVhbCgpYCBmdW5jdGlvbiB3b3VsZCBhcHBseSB0aGUgZXF1YXRpb25zIGluIGEgdmVjdG9yaXplZCB3YXkgZm9yIGVhY2ggY2h1bmsgb2Ygc3RlbXMgc2hhcmluZyB0aGUgc2FtZSBhbGxvbWV0cnkuIFRoZSBhbGxvbWV0cnkgYW5kIHRoZSBsZXZlbCBvZiBlc3RpbWF0aW9uIGlzIHBhcnQgb2YgdGhlIG91dHB1dCBvZiB0aGlzIGZ1bmN0aW9uLCBhcyBpdCBpcyBwb3RlbnRpYWxseSB1c2VmdWwgbWV0YWRhdGEgKGUuZy4gdG8gcmVwb3J0IG1ldGhvZHMgaW4gcHVibGljYXRpb25zKS4NCg0KVGhlIGBhbGxvbWV0cmllc2AgYXJndW1lbnQgaXMgdGhlIG91dHB1dCBmcm9tIHRoZSBgZ2V0X2FsbG9tZXRyeSgpYCBmdW5jdGlvbi4gU2luY2UgYWxsb21ldHJpZXMgYXJlIHN0b3JlZCBhcyByYXcgdGV4dCwgYSBzaW1wbGUgY29tYmluYXRpb24gb2YgYHBhcnNlYCBhbmQgYGV2YWxgIHNob3VsZCB3b3JrLg0KDQpgYGB7cn0NCg0KYmlvbWFzc19wZXJfaW5kaXZpZHVhbCA8LSBmdW5jdGlvbih4LCBhbGxvbWV0cmllcykNCnsNCiAgICAjIHRoZW4gZG8gdGhpczoNCiAgICBhbGxvbWV0cmllcy5pZCA8LSB1bmlxdWUoYWxsb21ldHJpZXNbLCAiYWxsb21ldHJ5Il0pDQogICAgYWdiIDwtIHJlcChOQSwgbnJvdyh4KSkNCiAgICBmb3IoYSBpbiBhbGxvbWV0cmllcy5pZCkNCiAgICB7DQogICAgICAgICBpIDwtIHdoaWNoKGFsbG9tZXRyaWVzWywgImFsbG9tZXRyeSJdID09IGEpDQogICAgICAgICBhZ2JbaV0gPC0gZXZhbChwYXJzZSh0ZXh0ID0gYSksIGVudmlyID0geFtpLF0pDQogICAgfQ0KICAgIGF0dHIoYWdiLCAiYWxsb21ldHJpZXMiKSA8LSBhbGxvbWV0cmllcw0KICAgIHJldHVybihhZ2IpDQp9DQoNCmBgYA0KDQoNCg0KIyMgRXhhbXBsZQ0KDQpgYGB7cn0NCmxpYnJhcnkoYmNpZXgpDQoNCnggPC0gYmNpMTJzMW1pbmkNCnggPC0geFshaXMubmEoeCRkYmgpLF0NCngkZGJoW2MoMywgNSwgNywgOSldIDwtIE5BDQp4JFN0ZW1JRCA8LSBwYXN0ZTAoeCR0cmVlSUQsICItIiwgeCRzdGVtSUQpDQp4JEQgPC0geCRkYmgNCngkSCA8LSBOQQ0KeCRIWzI6NV0gPC0gMjo1ICogMTANCngkV1NHIDwtIHJ1bmlmKG5yb3coeCksIDAuMywgMSkNCg0KQ1RGUy5FIDwtIGMoYmNpID0gMC41LCBwYXNvaCA9IDAuNywgeHh4eCA9IDEuNSkNCg0KYSA8LSBnZXRfYWxsb21ldHJ5KHggPSB4LCBzaXRlSUQgPSBOVUxMLCBFID0gTlVMTCwgc2l0ZV9sZXZlbF9hbGxvbWV0cnkgPSBOVUxMLCBzcGVjaWVzX2xldmVsX2FsbG9tZXRyeSA9IE5VTEwsIHN0ZW1fbGV2ZWxfYWxsb21ldHJ5ID0gTlVMTCkNCmhlYWQoYSwgMTApDQoNCmEgPC0gZ2V0X2FsbG9tZXRyeSh4ID0geCwgc2l0ZUlEID0gImJjaSIsIEUgPSBOVUxMLCBzaXRlX2xldmVsX2FsbG9tZXRyeSA9IE5VTEwsIHNwZWNpZXNfbGV2ZWxfYWxsb21ldHJ5ID0gTlVMTCwgc3RlbV9sZXZlbF9hbGxvbWV0cnkgPSBOVUxMKQ0KaGVhZChhLCAxMCkNCg0KYSA8LSBnZXRfYWxsb21ldHJ5KHggPSB4LCBzaXRlSUQgPSBOVUxMLCBFID0gNzc3LCBzaXRlX2xldmVsX2FsbG9tZXRyeSA9IE5VTEwsIHNwZWNpZXNfbGV2ZWxfYWxsb21ldHJ5ID0gTlVMTCwgc3RlbV9sZXZlbF9hbGxvbWV0cnkgPSBOVUxMKQ0KaGVhZChhLCAxMCkNCg0KcyA8LSBjKCIyICsgRCIpDQphIDwtIGdldF9hbGxvbWV0cnkoeCA9IHgsIHNpdGVJRCA9ICJiY2kiLCBFID0gTlVMTCwgc2l0ZV9sZXZlbF9hbGxvbWV0cnkgPSBzLCBzcGVjaWVzX2xldmVsX2FsbG9tZXRyeSA9IE5VTEwsIHN0ZW1fbGV2ZWxfYWxsb21ldHJ5ID0gTlVMTCkNCmhlYWQoYSwgMTApDQoNCnMxIDwtIGMoIjIgKyBEIikNCnMyIDwtIGxpc3QoYygiMiArIEQiLCAiMiAqIEQgKyBIIiksIGMoIjMgKiBEIikpDQphIDwtIGdldF9hbGxvbWV0cnkoeCA9IHgsIHNpdGVJRCA9ICJiY2kiLCBFID0gTlVMTCwgc2l0ZV9sZXZlbF9hbGxvbWV0cnkgPSBzMSwgc3BlY2llc19sZXZlbF9hbGxvbWV0cnkgPSBzMiwgc3RlbV9sZXZlbF9hbGxvbWV0cnkgPSBOVUxMKQ0KaGVhZChhLCAxMCkNCg0KczEgPC0gYygiMiArIEQiKQ0KczIgPC0gbGlzdChhcGVpbWUgPSBjKCIyICsgRCIsICIyICogRCArIEgiKSwgcXVhcmFzID0gYygiMyAqIEQiKSkNCmEgPC0gZ2V0X2FsbG9tZXRyeSh4ID0geCwgc2l0ZUlEID0gImJjaSIsIEUgPSBOVUxMLCBzaXRlX2xldmVsX2FsbG9tZXRyeSA9IHMxLCBzcGVjaWVzX2xldmVsX2FsbG9tZXRyeSA9IHMyLCBzdGVtX2xldmVsX2FsbG9tZXRyeSA9IE5VTEwpDQpoZWFkKGEsIDEwKQ0KDQoNCiMgVGhpcyBpcyB0aGUgbW9zdCB0eXBpY2FsIGNhc2U6DQphIDwtIGdldF9hbGxvbWV0cnkoeCA9IHgsIHNpdGVJRCA9ICJiY2kiLCBFID0gTlVMTCwgc2l0ZV9sZXZlbF9hbGxvbWV0cnkgPSBOVUxMLCBzcGVjaWVzX2xldmVsX2FsbG9tZXRyeSA9IE5VTEwsIHN0ZW1fbGV2ZWxfYWxsb21ldHJ5ID0gTlVMTCkNCmhlYWQoYSwgMTApDQoNCmFnYi5wZXIuaSA8LSBiaW9tYXNzX3Blcl9pbmRpdmlkdWFsKHgsIGFsbG9tZXRyaWVzID0gYSkNCmhlYWQoYWdiLnBlci5pLCAxMCkNCg0KYGBgDQoNCg==