Technical Memorandum

15375 SE 30th Place, Suite 250 October 2015

Bellevue, WA 98007

Review of: Exposure to Runoff from Coal-Tar-Sealed

Pavement Induces Genotoxicity and Impairment of DNA
Repair Capacity in the RTL-W1 Fish Liver Cell Line

Authors: Aude Kienzler, Barbara J. Mahler, Peter C. Van Metre, Nathalie Schweigert,
Alain Devaux, Sylvie Bony

Reference: Science of the Total Environment, 2015, 520:7380

Key Points
Runoff from refined tar-sealer-coated pavement increases DNA damage in
fish liver cells only at high concentrations with simultaneous exposure to
ultraviolet light.
Methodological shortcomings limit the applicability and extension of these
results to real-world environments.
Statistical flaws suggest that only the largest effect sizes may be statistically
significant.

Overview
Kienzler et al. (2015) investigates the genotoxicity of refined tar sealer (RTS) from pavement
runoff using in-vitro exposure to fish liver cells, assessed using the single-cell gel
electrophoresis (comet) assay to detect single and double strand breaks in DNA, as well as
inhibition of DNA repair. The study finds that significantly more DNA breaks than the control
occur only when tissues are exposed to both RTS and UVA. DNA repair appears to be equally
inhibited by RTS exposure with and without co-exposure to ultraviolet A radiation (UVA). The
methods used in the Kienzler et al. (2015) study appear to follow standard accepted methods;
however, there are some methodological shortcomings. The paper omits some details of the
analysis that could influence certain aspects of the findingsnamely, a potentially flawed
statistical analysisbut these flaws are unlikely to substantially change the main findings of the
paper.

Critique of Methods
Field Methods
The simulated runoff reported in this study was also used in at least two related studies (Mahler
et al. 2015; Mahler et al. 2014), and the following methodological critique draws from all three

1506377.000 - 3599
1
 Technical Memorandum
October 2015

studies to clarify sampling methods. Simulated runoff was generated for three types of test plots:
unsealed asphalt, RTS-covered asphalt, and Asphalt/RTS blend (ARB) (7% RTS) covered
asphalt. Traffic and parking activities were allowed after the recommended 24-hr curing period.
Runoff was simulated by sprinkling control water (an artificial rainwater mixture of well
water and deionized water from the toxicity-testing laboratory) across the plot and collecting it
at the downslope end of the pavement. A new area of the plot was sampled for each time point.
Simulated runoff from RTS was collected at 5 hrs and up to 111days after application, following
a 24-hr curing period. Simulated runoff of ARB was collected at 4 hrs and up to 36 days after
application.

The creation of simulated runoff samples and control solutions was not replicated, thus
excluding possibly relevant environmental variation in treatments. This lack of replication is
practiced in laboratory exposure experiments where solutions are created under uniform
conditions. In this case, however, environmental exposure was a condition of the experiment
and should have necessitated replication. These concerns would not affect the conclusion of the
study, but do reflect limited applicability of the results to other environmental or climatic
conditions.

Simulated runoff was collected on the plots for 7 days until a 2-day rain event occurred.
However, no samples of the runoff from this actual rain event were collected. This omission
would seem to represent a missed opportunity to validate the simulated runoff samples. In
addition, it is not clear how much washing occurred due to that natural rain event.

A more realistic exposure scenario would have been to use runoff from the plots after
subsequent rain events to determine the decrease in mobilized PAHs. Receiving streams would
likely observe a high-PAH pulse after the first rain event, followed by decreased PAH
concentrations from subsequent rain events.

The unsealed asphalt plot was used as a control for traffic and parking activities; however, in the
current study, the stock solution of artificial rainwater was used as a control, while data from the
unsealed control was not reported. The unsealed plot would be the more appropriate control, but
would still not actually be an entirely appropriate control for the sealcoated pavements. Auto-
related contamination (e.g., oil) would likely be absorbed by or adhere to unsealed pavement to
a greater extent than to sealed pavement. The stock solution of artificial rain used as a no-dose
control, and the simulated runoff treatments, are unlikely to have been representative of the
runoff to which aquatic organisms would be exposed in the natural environment, where runoff
would be subject to alteration by biotic activity and by adsorption to sediments and particulate
matter.

Incorporating a more realistic control sample would most likely have the effect of increasing the
DNA damage observed in the control, thus reducing the induction factor (effect size of the
treatment), and possibly reducing the number of treatments that are significantly different from
controls. The large effect size in some treatments, particularly those of the 10% RTSbased
runoff with UVA, would likely still be significantly higher than the modified controls.

1506377.000 - 3599
2
 Technical Memorandum
October 2015

Laboratory Methods
The experiment exposes cultured rainbow trout liver cells (RTL-W1) to RTS runoff water by
replacing 10% of the water used in a typical cell-culture medium with sampled runoff (or
control) water. The selected cell culture is commonly used in in vitro genotoxic assessments,
and the researchers sufficiently support their ability to maintain this culture under laboratory
conditions.

While the researchers report controlling for the osmotic differences, it is unclear to what extent
this occurred. Due to differences in treatment between the exposed runoff and the control water,
it is possible that atmospheric or other sources of chemical deposition (i.e., traffic and parking)
on the test plots could have created different osmotic properties in the treatment medium than in
the control medium, which may have interacted to affect the results. However, given that DNA
damage in many treatments did not differ significantly from the controls, this may be unlikely.

Comet assays are a widely used and accepted method of evaluating the genotoxic potential of a
wide variety of chemicals. Nonetheless, the bench-top and analysis procedures are not yet fully
standardized (Frenzilli et al. 2009), and the current study does not adequately explain some
aspects of the study. In a comet assay, damaged DNA is measured by observing the migration of
DNA fragments away from the main nucleus of a cell during gel electrophoresis. When
observing the DNA under a fluorescent light, the migrating DNA fragments look like a
somewhat dimmer tail extending from a brighter head where large, intact DNA molecules
are found nearest the nucleus. Numerous methods exist to quantify the degree of DNA damage
by comparing the amount of DNA found in the tail and the amount found in the head.

In the current study, the main measurement of DNA damage and repair is the ratio of the tail
florescence to head florescence (%TI). The paper does not discuss why this indicator was
chosen, as opposed to the many others available. Kienzler et al. (2015) claims %TI has a linear
relationship to the number of DNA strand breaks, but this is not common knowledge, nor are
data or one or more citations provided to support this claim. Two types of comet assays were
performed in the Kienzler et al. (2015) studya standard comet assay, which detects strand
breaks, and an FPG-modified comet assay, which shows additional DNA damage due to
oxidized and alkylated bases. The results of the standard assay are not provided for review. The
exclusion of these results is irregular, considering that they are reported in the methods.

Several important considerations for conducting comet assays are not clearly discussed in the
paper. First, it is critical that the cells chosen for scoring be selected at random from the many
hundreds of thousands typically present in a culture. The authors do not indicate how this
randomization was achieved. It is also important that the scoring be conducted anonymously.
That is, the scorer should have no knowledge of the treatment they are scoring; otherwise the
selection of cells may be unconsciously biased. The authors indicate that these procedures were
computerized, but the level of automation in cell selection and the extent to which automation
reduces these biases is not clear.

Finally, UVA is known to cause DNA fragmentation, and because it was of particular interest in
this study, extreme precautions should have been taken to prevent UVA exposure at all times,
except during experimental exposure (Tice et al. 2000). There is no indication that such
precautions were taken.

1506377.000 - 3599
3
 Technical Memorandum
October 2015

The issues mentioned above may have unintentionally affected the reported results via biased
selection of cells for scoring and lack of controls to ensure consistent replication of methods
across treatments. The extent of these issues is difficult to determine, because the appropriate
steps may have been taken but not reported. Kienzler et al. (2015) reports the methods in a
manner consistent with the level of detail commonly found in the literature on genotoxic assays.
The extent of the bias created by any of the omissions discussed above is unknown.

Statistical Analysis
The basic data values used in the statistical analysis are the %TIthe ratio of the amount of
DNA in the tail of the comet to the amount of DNA in its head. This measure has particular
statistical properties that are not directly amenable to parametric statistical analysis. The authors
overcome this limitation in a generally suitable way through the use of the non-parametric
Mann-Whitney test and the Holm-Bonferonii correction for multiple hypotheses. Each treatment
(sealant type [two levels] time point [five levels, including control] UV exposure [two
levels]) is performed in triplicate, and each triplicate is subsampled twice. This degree of
replication is appropriate and typical in genotoxicity studies performed with the comet assay.
However, several statistical issues may affect the significance of some reported results.

The first statistical issue is what appears to be confusion about how to aggregate subsamples and
samples to appropriately represent replication in each treatment. Each treatment is performed in
triplicate, and each of the triples should be considered a sample that is aggregated over the two
duplicate slides; however, the authors appear to calculate the mean and standard error of the
mean (SEM) for each treatment using all six slides, which is a form of pseudo-replication. The
duplicates of each triple should have been aggregated, and the means and SEM calculated using
these aggregates. The use of all six slides in each treatment appears to have continued when
calculating the Mann-Whitney statistic, which would artificially increase the number of
significant results detected.

The second statistical issue arises from the correction for multiple comparisons of the Mann-
Whitney statistics, which is a pair-wise test. In the described analysis, each treatment and time
point is tested separately for a significant difference from the control, and then the total number
of significant differences is corrected for the lack of independence arising from using the same
data to test multiple hypotheses (which would otherwise dramatically increase the probability of
detecting a significant result only by chance). Thus, when more hypotheses are tested with a
given set of data, the bar has to be higher for a given result to be considered significant. In the
current study, this bar height is corrected using the Holm-Bonferonii method, but it appears to
not have been corrected appropriately. That is, with only one control for all treatments over
time, 16 hypotheses would be tested per control. It appears that the authors conducted their
correction per treatment, meaning that they calculated a correction for only four hypotheses. The
exact calculation method is not reported, but the fact that the 7-day UVA treatment in Figure 4d
is considered a significant effect, while the larger effect in the 4-hr UVA treatment of Figure 4c
is not significant, seems to indicate that these tests were conducted separately.

Finally, the authors do not test for significant differences between the UVA and non-UVA
treatments. While some differences between UVA treatments are obvious from the figures,
some differences are not so obvious. Nonetheless, the photo-activation of toxicity caused by the
presence of UVA light during PAH exposure is a major finding of the study, and should have

1506377.000 - 3599
4
 Technical Memorandum
October 2015

been tested specifically. Furthermore, the study reveals that toxicity occurs only during co-
exposure to UVA light. While the quantity of UVA exposure is consistent with measurements
made in typical surface waters, it is not likely consistent with the UVA light to which liver cells
are exposed in vivo, nor is there any indication that similar genotoxicity would occur in tissues
(i.e., skin, eyes, and possibly gills) that would be exposed to prescribed levels of UVA light.

These concerns regarding statistical treatment are important for the detailed understanding of
results. The overall large effect size detected at the 10% RTS treatment level can, however, be
understood without statistical analysis.

Conclusion
The study by Kienzler et al. provides evidence of genotoxicity of runoff from RTS treatment
surfaces. The doses tested in this experiment are likely higher than environmentally relevant
doses, due to the lack of biotic and abiotic interactions likely to occur in a natural setting. Some
methodological omissions and statistical flaws indicate the possibility that the effect size may
not be as large as reported, and limit the ability to apply these results to other, more
environmentally relevant situations. The available evidence does support further, more detailed
investigation.

References
Frenzilli, G., M. Nigro, and B.P. Lyons. 2009. The comet assay for the evaluation of genotoxic
impact in aquatic environments. Rev. Mutat. Res. 681:8092.

Mahler, B.J., C.G. Ingersoll, P.C. Van Metre, J.L. Kunz, and E.E. Little. 2015. Acute toxicity of
runoff from sealcoated pavement to Ceriodaphnia dubia and Pimephales promelas. Environ. Sci.
Technol. p.150410150020004. Available at:
http://pubs.acs.org/doi/abs/10.1021/acs.est.5b00933.

Mahler, B.J., P.C. Van Metre, and W.T. Foreman. 2014. Concentrations of polycyclic aromatic
hydrocarbons (PAHs) and azaarenes in runoff from coal-tar- and asphalt-sealcoated pavement.
Environ. Pollut. 188:8187. Available at: http://dx.doi.org/10.1016/j.envpol.2014.01.008.

Tice, R.R., E. Agurell, and D. Anderson. 2000. Single cell gel/comet assay: Guidelines for in
vitro and in vivo genetic toxicology testing. Environ. Molecular Mutagen. 35(3):206221.

1506377.000 - 3599
5