Replication Study: Melanoma exosomes educate bone marrow progenitor cells toward a pro-metastatic phenotype through MET

Abstract
Introduction
Results and discussion
Materials and methods
Data availability
References
Article and author information
Metrics

Abstract

As part of the Reproducibility Project: Cancer Biology we published a Registered Report (Lesnik et al., 2016) that described how we intended to replicate selected experiments from the paper ‘Melanoma exosomes educate bone marrow progenitor cells toward a pro-metastatic phenotype through MET’ (Peinado et al., 2012). Here we report the results. We regenerated tumor cells stably expressing a short hairpin to reduce Met expression (shMet) using the same highly metastatic mouse melanoma cell line (B16-F10) as the original study, which efficiently downregulated Met in B16F10 cells similar to the original study (Supplementary Figure 5A; Peinado et al., 2012). Exosomes from control cells expressed Met, which was reduced in exosomes from shMet cells; however, we were unable to reliably detect phosphorylated Met in exosomes. We tested the effect of exosome-dependent Met signaling on primary tumor growth and metastasis. Similar to the results in the original study, we did not find a statistically significant change in primary tumor growth. Measuring lung and femur metastases, we found a small increase in metastatic burden with exosomes from control cells that was diminished when Met expression was reduced; however, while the effects were in the same direction as the original study (Figure 4E; Peinado et al., 2012), they were not statistically significant. Differences between the original study and this replication attempt, such as level of knockdown efficiency, cell line genetic drift, sample sizes, study endpoints, and variability of observed metastatic burden, are factors that might have influenced the outcomes. Finally, we report meta-analyses for each result.

https://doi.org/10.7554/eLife.39944.001

Introduction

The Reproducibility Project: Cancer Biology (RP:CB) is a collaboration between the Center for Open Science and Science Exchange that seeks to address concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology (Errington et al., 2014). For each of these papers a Registered Report detailing the proposed experimental designs and protocols for the replications was peer reviewed and published prior to data collection. The present paper is a Replication Study that reports the results of the replication experiments detailed in the Registered Report (Lesnik et al., 2016) for a paper by Peinado et al., and uses a number of approaches to compare the outcomes of the original experiments and the replications.

In 2012, Peinado et al. reported that exosomes isolated from highly metastatic murine melanoma cells (B16-F10) increased the metastatic burden of the primary tumors compared to exosomes from poorly metastatic melanomas (B16-F1) or control liposomes. Hepatocyte growth factor receptor (Met) was identified as a highly expressed protein in B16-F10 exosomes. Reduction of Met, and phosphorylated Met (pMet) by shRNA in B16-F10 exosomes resulted in a statistically significant decrease in lung and bone metastatic burden compared to controls (Peinado et al., 2012). Exosomes derived from melanoma cells were proposed to promote metastasis by education of bone marrow-derived cells through horizontal transfer of exosomal Met in order to prime the pre-metastatic niche and increase vascularization (Peinado et al., 2012).

The Registered Report for the paper by Peinado et al. described the experiments to be replicated (Figure 4E and Supplementary Figures 1C and 5A), and summarized the current evidence for these findings (Lesnik et al., 2016). Since that publication, additional studies have reported in different models that tumor derived exosomes administered to mice prior to tumor cell injection increased metastatic burden by inducing pre-metastatic niche formation (Costa-Silva et al., 2015; Fong et al., 2015; Hoshino et al., 2015; Liu et al., 2016; Plebanek et al., 2017; Zhou et al., 2014). Additionally, using the same tumor model as Peinado et al., 2012, Met expression was found to be heterogeneous in B16-F10 cells, with higher lung metastatic burden, and lower primary tumor burden, in cells expressing high levels of Met compared to cells expressing low levels of Met (Adachi et al., 2016). Injection of exosomes from high Met expressing cells increased the metastatic burden of low Met expressing cells (Adachi et al., 2016). The molecules present on tumor-derived exosomes, such as the specific repertoire of integrins, are important in dictating metastatic organotropism (Hoshino et al., 2015). There have been numerous studies that have aimed to identify exosomal cargo (e.g. proteins) (Keerthikumar et al., 2016). Similar to the study by Peinado et al., 2012, other studies, using various techniques (e.g. proteomic profiling, reverse phase protein array), have identified MET in exosomes from melanoma cells (Lazar et al., 2015; Steenbeek et al., 2018) as well as in hepatocellular carcinoma cells (He et al., 2015), neuroblastoma cells (Keerthikumar et al., 2015), ovarian cancer cells (Liang et al., 2013), sera from healthy donors and prostate cancer patients (Cannistraci et al., 2017), 293 T cells (Li et al., 2016), and as a fusion protein (PTPRZ1-MET) in glioblastoma cells (Zeng et al., 2017), while other studies did not identify MET in exosomes from breast cancer patients (Chen et al., 2017) nor in some melanoma cells (Lazar et al., 2015).

The outcome measures reported in this Replication Study will be aggregated with those from the other Replication Studies to create a dataset that will be examined to provide evidence about reproducibility of cancer biology research, and to identify factors that influence reproducibility more generally.

Results and discussion

Generation and characterization of shMet B16-F10 cells and exosomes

To test the effect exosome-dependent Met signaling has on primary tumor growth and metastasis, we used the same highly metastatic mouse melanoma cell line (B16-F10) and the same lentiviral system as the original study to make B16-F10 cells stably expressing an shRNA targeting Met (shMet) or a control shRNA (shScr) using the same targeting sequences as the original study. The experimental approach to generate and characterize the stable cells and isolated exosomes was described in Protocol 1 and 2 of the Registered Report (Lesnik et al., 2016). We tested various multiplicity of infection (MOI) ratios, all of which displayed expression of the shRNA with corresponding decreased Met and Met levels in shMet cells compared to shScr cells (Figure 1—figure supplement 1). We planned to utilize cells generated with an MOI of 10, similar to the original study, but observed that the Met levels in the shScr cells at this MOI were, for unknown reasons, decreased when compared to the shScr cells generated at the other MOI ratios (Figure 1—figure supplement 1C). Thus, we proceeded with the stable cells generated with an MOI of 20, which had 22.6% Met expression, and 25.1% phosphorylated Met (pMet) expression in the shMet cells relative to shScr cells (Figure 1A–C). The stable cell lines generated in the original study were reported to have 64.1% Met expression and 23.4% pMet expression in the shMet cells relative to shScr cells (Peinado et al., 2012).

Figure 1 with 1 supplement see all

Download asset Open asset

Characterization of shMet B16-F10 cells and exosomes.

B16-F10 cells engineered to express shScr or shMet were used to purify exosomes. (A) Representative Western blots of exosomes and B16-F10 cells expressing the indicated shRNA were probed with antibodies specific for total Met (top panel) and Gapdh (bottom panel). Membranes were cut at ~75 kDa so that Met and Gapdh could be probed in parallel. Repeat indicates the number of independently isolated exosome and cell lysate preparations from the same batch of infected cells. The fourth lane, labeled ‘Cells’ are lysate from B16-F10 cells expressing shScr. (B) Representative Western blots of exosomes and B16-F10 cells expressing the indicated shRNA were probed with antibodies specific for phosphorylated (Tyr 1234/1235) Met (pMet) (top panel) and Gapdh (bottom panel). Membranes were cut at ~75 kDa so that pMet and Gapdh could be probed in parallel. Repeat indicates the number of independently isolated exosome preparations from the same batch of infected cells. (C) Western blot bands were quantified for cells. Met or pMet levels were normalized to Gapdh, and protein expression presented relative to shScr conditions. Expression level of shScr condition was assigned a value of 1. Means reported and error bars represent SD. Results are from 3 independent biological repeats for Met expression and 4 independent biological repeats for pMet expression. Exploratory analysis: one-sample t-test on Met levels (Met/Gapdh) in shMet cells compared to a constant of 1 (shScr cells): t(2) = 8.41, p = 0.014, Bonferroni corrected p = 0.028, Cohen’s d = 4.85, 95% CI [0.55, 9.43]; one-sample t-test on pMet levels (Met/Gapdh) in shMet cells compared to a constant of 1 (shScr cells): t(3) = 8.94, p = 0.003, Bonferroni corrected p = 0.006, Cohen’s d = 4.47, 95% CI [1.02, 8.01]. (D) Representative Western blot of exosomes isolated from cells expressing the indicated shRNA probed with exosome markers Hsc70, Tsg101, and Cd63 specific antibodies. Experiment performed on 3 independent biological repeats for each condition from the same batch of infected cells. Additional details for this experiment can be found at https://osf.io/aqm2m/.

https://doi.org/10.7554/eLife.39944.002

Using the same ultracentrifugation approach as the original study, we isolated exosomes from shScr and shMet cells. We confirmed the presence of the known exosome markers Hsc70, Tsg101, and Cd63 (Figure 1D) and found exosome number and size distribution quantified by NanoSight analyses to be similar between the two groups (Table 1). To test if Met and pMet were expressed in B16-F10 exosomes we performed Western blots and found that compared to shScr exosomes the amount of Met expressed in shMet exosomes was decreased (Figure 1A); however, we were unable to reliably detect pMet expression in exosomes (Figure 1B). Importantly, this was not due to the inability of the antibody to detect pMet by Western blot, since pMet was detected in cell lysates run on the same gel. Additionally, as a preventative measure to block any residual phosphatase activity, we added inhibitors when preparing the isolated exosomes for electrophoresis. However, if there were carryover phosphatases present during isolation of exosomes by ultracentrifugation the pMet levels could have been diminished before the inhibitors were added. Furthermore, the exosomes were prepared for electrophoresis after first being stored at −20˚C, instead of immediately after isolation, which might be critical for the detection of phospho-protein detection in exosomes. Inclusion of untransduced B16-F10 controls could have also indicated if unintended changes occurred in the stable cell lines utilized, especially given the potential impact puromycin selection can have on transcriptome profiles (Guo et al., 2017), and should be considered in the experimental design of future studies. The original study reported Met and pMet expression by Western blot in B16-F10 exosomes; however, the level of knockdown achieved in shMet exosomes was not reported. To summarize, we observed reduced Met and pMet expression in shMet cells compared to shScr cells, consistent with the stable cells reported in the original study; however, while we detected Met in exosomes from shScr cells, that was reduced in exosomes from shMet cells, we were unable to reliably detect pMet expression in isolated exosomes.

Table 1

Nanosight analysis of exosomes.

Summary of exosome number and size distribution, with or without the finite track length adjustment (FTLA) algorithm, quantified by NanoSight analysis. All values (mean, SD, median, span, concentration (particles/ml)) are given as averages for preps generated during this study (n = 15 per cell line).

https://doi.org/10.7554/eLife.39944.004

		Mean	Median	Sd	Span	Particles/mL
shMet	FTLA size distribution	91.07	72.9	62.89	1.253	6.253e+11
	size distribution	91	75	65.44	1.334	6.253e+11
shScr	FTLA size distribution	88.47	71.8	56.04	1.211	9.477e+11
	size distribution	88.67	73.67	61.8	1.321	9.477e+11

Exosome-dependent Met signaling on primary tumor growth and metastasis

We next used shScr and shMet exosomes to replicate an experiment that tested whether exosome-dependent Met signaling impacted primary tumor growth and metastasis. Synthetic unilamellar 100 nm liposomes were used as a control to test if tumor exosomes enhance metastasis. This experiment is similar to what was reported in Figure 4E of Peinado et al., 2012 and described in Protocol 3 in the Registered Report (Lesnik et al., 2016). Freshly isolated exosomes from shScr or shMet cells, or synthetic liposomes, were injected into C57BL/6 female mice three times a week for a total of four weeks, thereafter we implanted B16-F10 tumor cells engineered to express luciferase (B16-F10-luc). The planned study design involved waiting 21 days after B16-F10-luc tumor cell implantation before sacrificing the mice for analysis; however, two animals were found dead before this time point was reached (17 days after implantation), and of the surviving mice, the largest tumors reached >1 cm³ prompting us to stop the experiment early (18 days after implantation). We measured primary tumor growth during the length of the study and observed increased growth among all the conditions (Figure 2A, Figure 2—figure supplement 1A–D). Interestingly, the primary tumors in mice injected with shScr exosomes were on average smaller than the primary tumors in mice injected with shMet or synthetic liposomes. This was also observed when measuring the weight of the dissected primary tumors at the end of the study (Figure 2B). There are a number of factors that can affect tumor growth, such as availability of nutrients, oxygen, and space that influence initial and continued growth of the tumor (Cornelis et al., 2013; Talkington and Durrett, 2015). There was not a statistically significant difference between the tumor weights of the three groups (Kruskal-Wallis: H(2) = 2.85, p=0.24). Additionally, we conducted two planned comparisons (shScr vs shMet; shScr vs synthetic liposomes), which were not statistically significant (see Figure 2 figure legend). This is consistent with the original study that stated no differences in primary tumor growth between the three groups; however, those data were not shown, preventing direct comparison of the original study results and the results from this replication attempt.

Figure 2 with 1 supplement see all

Download asset Open asset

Primary tumor growth and metastatic burden of mice injected with exosomes.

Female C57BL/6 mice were subcutaneously injected with B16-F10-luciferase cells after 4 weeks of intravenous injections (3 times a week) of shScr exosomes, shMet exosomes, or synthetic liposomes. (A) Following primary tumor detection caliper measurements were taken 3 times a week and used to calculate tumor volume. Line graph of tumor volume (y-axis is natural log scale) with means reported and error bars representing s.e.m. Number of mice monitored per group: synthetic liposomes = 7, shScr = 7, shMet = 7. Of note, not all mice had detectable primary tumors at the first measurement, one of the mice injected with synthetic liposomes never formed a detectable primary tumor despite the presence of metastatic burden, and two mice (one injected with B16-F19 shScr exosomes and one injected with synthetic liposomes) were found dead before the end of the study (day 17). Individual mouse tumor volume data is reported in Figure 2 - figure supplement 1. (B) At the end of the experiment (day 18), primary tumors were excised and weighed. Dot plot with means reported as crossbars and error bars represent s.e.m. Number of primary tumor weights per group: synthetic liposomes = 5, shScr = 6, shMet = 7. Kruskal-Wallis test on all three groups: H(2) = 2.85, p = 0.24. Planned Wilcoxon-Mann-Whitney comparison between shScr and synthetic liposomes: U = 22, p = 0.247, Cliff’s d = 0.47, 95% CI [-0.28, 0.86]. Planned Wilcoxon-Mann-Whitney comparison between shScr and shMet: U = 32, p = 0.138, Cliff’s d = 0.52, 95% CI [-0.10, 0.85]. (C) Metastatic burden in lungs quantified by luciferin photon flux at 18 days after B16-F10-luc tumor cell injections. Box and whisker plot (y-axis is natural log scale) with median represented as the line through the box and whiskers representing values within 1.5 IQR of the first and third quartile. Individual data points represented as dots. Number of mice per group: synthetic liposomes = 6, shScr = 6, shMet = 7. One-way ANOVA on the luciferin photon flux values (natural log-transformed); F(2,16) = 0.226, uncorrected p = 0.800 with *a priori* alpha level of 0.025, Bonferroni corrected p > 0.99. Planned contrast between shScr and synthetic liposomes; Fisher’s LSD test; t(16) = 0.543, p = 0.594 with *a priori* alpha level of 0.025, Bonferroni corrected p > 0.99, Cohen’s d = 0.31, 95% CI [-0.83, 1.45]. Planned contrast between shScr and shMet; Fisher’s LSD test; t(16) = 0.620, p = 0.544 with *a priori* alpha level of 0.025, Bonferroni corrected p > 0.99, Cohen’s d = 0.34, 95% CI [-0.76, 1.44]. (D) Metastatic burden in femurs quantified by luciferin photon flux at 18 days after B16-F10-luc tumor cell injections. Box and whisker plot (y-axis is natural log scale) with median represented as the line through the box and whiskers representing values within 1.5 IQR of the first and third quartile. Individual data points represented as dots. Number of mice per group: synthetic liposomes = 6, shScr = 6, shMet = 7. One-way ANOVA on the luciferin photon flux values (natural log-transformed); F(2,16) = 0.190, uncorrected p = 0.829 with *a priori* alpha level of 0.025, Bonferroni corrected p > 0.99. Planned contrast between shScr and synthetic liposomes; Fisher’s LSD test; t(16) = 0.573, p = 0.575 with *a priori* alpha level of 0.025, Bonferroni corrected p > 0.99, Cohen’s d = 0.33, 95% CI [-0.82, 1.46]. Planned contrast between shScr and shMet; Fisher’s LSD test; t(16) = 0.105, p = 0.918 with *a priori* alpha level of 0.025, Bonferroni corrected p > 0.99, Cohen’s d = 0.06, 95% CI [-1.03, 1.15]. Additional details for this experiment can be found at https://osf.io/mzywk/.

https://doi.org/10.7554/eLife.39944.005

Metastatic burden was quantified by luciferin photon flux in the lungs and femurs of the mice from each group at the end of the study. Mice injected with the shScr exosomes achieved an average of 1.0×10⁵ photons/sec in the lungs, which was reduced to 4.2×10⁴ photons/sec in mice injected with shMet exosomes and 5.3×10⁴ photons/sec in mice injected with synthetic liposomes (Figure 2—figure supplement 1E). Thus, the group means of the natural log transformed lung metastatic burden data were 10.8, 10.4, and 10.5 log units, respectively (Figure 2C). On average, there was a 36.9% natural log based percentage difference in lung metastatic burden between mice injected with shScr exosomes and shMet exosomes, and a 33.6% difference between mice injected with shScr and synthetic liposomes. To test if shMet exosomes or synthetic liposomes had decreased metastatic burden compared to shScr exosomes, we performed a one-way ANOVA on the luciferin photon flux values (natural log-transformed), which was not statistically significant (F(2,16) = 0.226, uncorrected p=0.800, Bonferroni corrected p>0.99). Additionally, we conducted two planned comparisons (shScr vs synthetic liposomes; shScr vs shMet), which were not statistically significant (see Figure 2 figure legend). The original study reported metastatic burden in the lungs of mice injected with shScr exosomes [M = 5.1×10⁴ photons/sec; 10.8 log units] were reduced by 165.3% [M = 1.3×10⁴ photons/sec; 9.1 log units] and 80.0% [M = 2.3×10⁴ photons/sec; 10.0 log units] with shMet exosomes and synthetic liposomes, respectively (Peinado et al., 2012). The range of metastatic lung values reported in the original study had a relative standard deviation (RSD) that were smaller (shScr = 2.9%; shMet = 9.9%; synthetic liposome = 3.8%) than the RSD observed in this replication attempt (shScr = 13%; shMet = 7.1%; synthetic liposome = 9.7%), particularly among the shScr and synthetic liposome groups.

For femurs, the average metastatic burden of mice injected with the shScr exosomes achieved an average of 7.4×10⁵ photons/sec (12.2 log units), which was reduced to 3.3×10⁵ photons/sec (12.1 log units) in mice injected with shMet exosomes and 1.9×10⁵ photons/sec (11.7 log units) in mice injected with synthetic liposomes (Figure 2D, Figure 2—figure supplement 1F). So, on average, there was a 7.8% natural log based percentage difference in femur metastatic burden between mice injected with shScr exosomes and shMet exosomes, and a 44.3% difference between mice injected with shScr and synthetic liposomes, which was not statistically significant (see Figure 2 figure legend). While the original study reported metastatic burden in the femurs of mice injected with shScr exosomes [M = 4.5×10⁴ photons/sec; 10.4 log units], there was no reported metastatic burden in the femurs of mice injected with shMet exosomes or synthetic liposomes (Peinado et al., 2012). Additionally, similar to the lung values, the RSD for the metastatic femur values were smaller in the original study (shScr = 8.4%; shMet = 0%; synthetic liposome = 0%) than this replication attempt (shScr = 14%; shMet = 10%; synthetic liposome = 8.4%).

The higher RSD observed in this study is one of the factors that could influence if statistical significance is reached, particularly since the sample size of this replication attempt was determined a priori to detect the effect based on the originally reported data. Importantly, these results should take into consideration the experimental endpoint of this replication attempt, which was shorter than the original study and what was indicated in the Registered Report (18 days instead of 21 days), which could account for less macro-metastasis. Also, as mentioned above, inclusion of untransduced B16-F10 controls could have indicated if unintended changes to the cargo of exosomes from the stable cell lines utilized occurred and should be considered in the experimental design of future studies. To summarize, for this experiment we found results that were in the same direction as the original study and not statistically significant.

Meta-analysis of original and replication effects

We performed a meta-analysis using a random-effects model, where possible, to combine each of the effects described above as pre-specified in the confirmatory analysis plan (Lesnik et al., 2016). To provide a standardized measure of the effect, a common effect size was calculated for each effect from the original and replication studies. Cliff’s delta (d) is a non-parametric estimate of effect size that measures how often a value in one group is larger than the values from another group. It is used in this case because of violations to the assumptions of normality and equal variance in the original and replication studies. Importantly, the confidence interval around Cliff’s d is asymmetric, while the p value is calculated using the normal distribution and is thus not well defined; however, there is no agreement on how to compute p values from an asymmetric distribution (Dunne et al., 1996; Rohatgi and Saleh, 2000). The estimate of the effect size of one study, as well as the associated uncertainty (i.e. confidence interval), compared to the effect size of the other study provides another approach to compare the original and replication results (Errington et al., 2014; Valentine et al., 2011). Importantly, the width of the confidence interval for each study is a reflection of not only the confidence level (e.g. 95%), but also variability of the sample (e.g. SD) and sample size.

There were a total of four comparisons made with the metastatic burden data from lungs and femurs of mice injected with shScr exosomes, shMet exosomes, or synthetic liposomes (Figure 3). For all comparisons, the results of the original study and this replication were consistent when considering the direction of the effect; however, the point estimate of the replication effect size was not within the confidence interval of the original result, and vice versa. The meta-analyses were not statistically significant for any of the effects (see Figure 3 figure legend). Furthermore, the large confidence intervals of the meta-analyses along with statistically significant Cochran’s Q tests (lung: shScr and liposomes, p=0.045; shScr and shMet, p=0.020; femur: shScr and liposomes, p=0.0084; shScr and shMet, p=0.0046) suggest heterogeneity between the original and replication studies.

Figure 3

Download asset Open asset

Meta-analyses of each effect.

Effect size (Cliff’s delta) and 95% confidence interval are presented for Peinado et al., 2012, this replication attempt (RP:CB), and a meta-analysis to combine the two effects. Cliff’s delta is a standardized measure of how often a value in one group is larger than the values from another group, with a larger positive value indicating a decrease in metastatic burden in animals injected with either shMet exosomes or liposomes compared to animals injected with shScr exosomes. Random effects meta-analysis: lung metastatic burden (shScr v. liposome, p = 0.149; shScr v. shMet, p = 0.226) and femur metastatic burden (shScr v. liposome, p = 0.264; shScr v. shMet, p = 0.317). Sample sizes used in Peinado et al., 2012 and this replication attempt are reported under the study name. Additional details for these meta-analyses can be found at https://osf.io/c69jx/.

https://doi.org/10.7554/eLife.39944.007

This direct replication provides an opportunity to understand the present evidence of these effects. Any known differences, including reagents and protocol differences, were identified prior to conducting the experimental work and described in the Registered Report (Lesnik et al., 2016). However, this is limited to what was obtainable from the original paper and through communication with the original authors, which means there might be particular features of the original experimental protocol that could be critical, but unidentified. So while some aspects, such as cell line, mouse strain, and technique for exosome isolation were maintained, others were changed during the execution of the replication that could affect results, such as the time from cell injection after exosome education until euthanasia, which was shorter in this replication attempt than what was conducted in the original study (18 days instead of 21 days). Additionally, others were unknown or not easily controlled for. These include variables such as cell-intrinsic changes in Met expression and the associated gene expression profiles (Adachi et al., 2016), cell line genetic drift (Ben-David et al., 2018; Hughes et al., 2007; Kleensang et al., 2016), non-random genetic/transcriptional drift in the heterogeneous stable cells (Guo et al., 2017; Shearer and Saunders, 2015), genetic heterogeneity of mouse inbred strains (Casellas, 2011), the microbiome of recipient mice (Macpherson and McCoy, 2015), housing temperature in mouse facilities (Kokolus et al., 2013), and lot variability and quality of key reagents, such as the lentiviral particles (Leek et al., 2010). Environmental differences such as husbandry staff, bedding type and source, light levels, and other intangibles, all of which, by necessity, differed between the studies also affect experimental outcomes with mice (Howard, 2002; Jensen and Ritskes-Hoitinga, 2007; Nevalainen, 2014; Sorge et al., 2014). The difference in sample sizes between the studies is also a factor that could limit the opportunity to detect a statistically significant difference, especially considering the higher variability observed in this replication attempt, as described above. The difference in achieved knockdown of Met in exosomes between the original study and this replication attempt is another important factor to consider. A higher level of knockdown might be required to yield a given phenotype with this experimental design. Importantly, observing and reporting all outcomes are informative to establish the range of conditions under which a given phenotype can be observed (Bailoo et al., 2014). Whether these or other factors influence the outcomes of this study is open to hypothesizing and further investigation, which is facilitated by direct replications and transparent reporting.

Materials and methods

Key resources table

Reagent type (species) or resource	Designation	Source or reference	Identifiers	Additional information
Strain, strain background (Mus musculus, C57BL/6, female)	C57BL/6	Charles River	Strain code: 027; RRID: IMSR_CRL:27
Cell line (M. musculus)	B16-F10	doi:10.1038/nm.2753	RRID: CVCL_0159
Cell line (M. musculus)	B16-F10-luc	doi:10.1038/nm.2753		expresses firefly luciferase
Other	synthetic uniloamellar 100 nm liposomes	Encapsula NanoSciences		13 mg/ml L-a-Phosplatydilcholine, 2.78 mg/ml cholesterol (7:3 molar ratio P:C)
Sequence-based reagent	pGIPZ non-silencing shRNA lentiviral control particles	Dharmacon/GE Life Sciences	cat# RHS4348	sense sequence: 5’-ATCTCGCTTGGGCGAGAGTAAG-3’
Sequence-based reagent	pGIPZ mouse Met shRNA lentiviral particles	Dharmacon/ GE Life Sciences	cat# VGM5520- 200377256; clone ID: V3LMM_ 456078	sense sequence: 5’-CCAGACTTTTCATACAAGA-3’
Antibody	mouse anti-Hsc70	Enzo Life Sciences	cat# ALX-804–067; RRID: AB_10538284	1:500 dilution in SuperBlock, O/N at 4 ˚C
Antibody	mouse anti-Tsg101	Santa Cruz Biotechnology	cat# sc-7964; RRID: AB_671392	1:500 dilution in SuperBlock, O/N at 4 ˚C
Antibody	rabbit anti-Cd63	System Biosciences	cat# EXOAB-CD63A-1; RRID:AB_2561274	1:1000 dilution in SuperBlock, O/N at 4 ˚C
Antibody	mouse anti-Met	Cell Signaling Technology	cat# 3127; RRID:AB_331361	1:1000 dilution in SuperBlock, O/N at 4 ˚C
Antibody	rabbit anti-pMet (Tyr 1234/5)	Cell Signaling Technology	cat# 3077; RRID:AB_2315156	1:1000 dilution in SuperBlock, O/N at 4 ˚C
Antibody	rabbit anti-Gapdh	Santa Cruz Biotechnology	cat# sc-25778; RRID:AB_10167668	1:1000 dilution in SuperBlock, O/N at 4 ˚C
Antibody	rabbit anti-pMet (Tyr 1234/5)	Thermo Fisher Scientific	cat# MA5-15083; RRID:AB_10983015	1:1000 dilution in SuperBlock, O/N at 4 ˚C
Antibody	HRP-conjugated sheep anti-mouse	GE Healthcare	cat# NA931; RRID:AB_772210	1:15,000 dilution in SuperBlock, 1 hr at RT
Antibody	HRP-conjugated donkey anti-rabbit	GE Healthcare	cat# NA934; RRID:AB_772206	1:15,000 dilution in SuperBlock, 1 hr at RT
Sequence-based reagent	shMet primer	this paper		Forward: 5’-CCAGACTTT TCATACAAGAATA-3’
Sequence-based reagent	universal reverse primer	QuantiMir Kit, System Biosciences	cat# RA420A-1
Sequence-based reagent	U6 primer	LncRNA Profiler qPCR Array Kit, System Biosciences	cat# RA900A-1
Sequence-based reagent	Met primer	this paper		Forward: 5’-CTGGACA GATTGTGGGAGTAAG −3’
Sequence-based reagent	Gapdh primer	LncRNA Profiler qPCR Array Kit, System Biosciences	cat# RA900A-1
Software, algorithm	ImageJ	doi:10.1038/nmeth.2089	RRID:SCR_003070	version 1.49 u
Software, algorithm	SDS	Applied Biosystems	RRID:SCR_015806	version 2.4
Software, algorithm	Nanoparticle tracking Analysis (NTA)	Nanosight	RRID:SCR_014239	version 2.3
Software, algorithm	Living Image	Caliper Life Sciences	RRID:SCR_014247	version 4.4
Software, algorithm	R Project for statistical computing	https://www.r-project.org	RRID:SCR_001905	version 3.5.1

As described in the Registered Report (Lesnik et al., 2016), we attempted a replication of the experiments reported in Figure 4E and Supplementary Figures 1C and 5A of Peinado et al., 2012. A detailed description of all protocols can be found in the Registered Report (Lesnik et al., 2016) and are described below with additional information not listed in the Registered Report, but needed during experimentation.

Cell culture

Request a detailed protocol

B16-F10 (RRID:CVCL_0159) and B16-F10-luciferase (expressing firefly luciferase) (shared by Dr. David Lyden, Weill Medical College of Cornell University) were maintained in DMEM (Thermo Fisher Scientific, cat# MT10013CV) supplemented with 10% exosome-depleted fetal bovine serum (FBS) and 100 U/ml penicillin-streptomycin (Thermo Fisher Scientific, cat# 15-140-122) at 37°C in a humidified atmosphere at 5% CO₂. FBS (Hyclone, cat# SH30088.03) was depleted of bovine exosomes by ultracentrifugation (Beckman Coulter, Optima L-90K) at 100,000xg (Beckman Coulter, Type 70 Ti Rotor) for 70 min at 4°C. Quality control data for both cell lines are available at https://osf.io/3x58z/. This includes results confirming the cell lines were free of mycoplasma contamination as well as STR DNA profiling of the cell lines (B16-F10 cells: DDC Medical, Fairfield, Ohio; B16-F10-luciferase cells: IDEXX BioResearch, Columbia, Missouri), which were confirmed to be the indicated cell lines when queried against STR profile databases. Additionally, B16-F10-luciferase cells were confirmed free of common mouse pathogens (IDEXX BioResearch).

Stable cell generation

Request a detailed protocol

B16-F10 cells (2.5×10⁴ cells/well of 24 well plate) grown in culture medium supplemented with 1X TransDux were transduced with either pGIPZ mouse Met shRNA lentiviral particles (Dharmacon/GE Life Sciences, cat# VGM5520-200377256, clone ID: V3LMM_456078, lot# V16042101, sense sequence: 5’-CCAGACTTTTCATACAAGA-3’) or pGIPZ non-silencing shRNA lentiviral control particles (Dharmacon/GE Life Sciences, cat# RHS4348, lot# 150529606, sense sequence: 5’-ATCTCGCTTGGGCGAGAGTAAG-3’) at MOIs of 10, 20, or 50, incubated overnight (16 hr), then growth medium was replaced. Transduction efficiency was measured by GFP expression 72 hr later and then transduced cells were selected with 1.5 µg/ml puromycin for 28 days. Cells resistant to puromycin were checked for GFP expression. Growth rates were similar among all stable cell populations generated based on qualitative observations documented during passaging. Cells were frozen down and stored for further use. Thawed cells were maintained in 1.5 µg/ml puromycin, except when cells were plated for experiments. Microscopy images of cells including laboratory notes on cell culturing of stable cell lines are available at https://osf.io/yp5g6/.

Exosome purification and tracking analysis

Request a detailed protocol

Supernatant from B16-F10 shMet and B16-F10 shScr cells collected 48–72 hr after initial plating (two 15 cm plates at ~5×10⁶ cells/plate) were pelleted at 500xg (Beckman Coulter, Allegra X-14R) for 10 min at 4°C. The supernatant was clarified by ultracentrifugation (Beckman Coulter, Optima L-90K) at 20,000xg (Beckman Coulter, Type 70 Ti Rotor) for 20 min at 4°C. Exosomes were then collected by ultracentrifugation at 100,000xg for 70 min at 4°C. Protein concentration of exosomes were determined with a BCA assay using a standard curve (data available at https://osf.io/9xnbf/) and either suspended in PBS and used directly for experiments (in vivo animal experiments) or stored at −20°C until analysis (Western blots and Nanosight). Characterization of exosomes was performed with a nanoparticle analysis system (Nanosight, LM10) equipped with a blue laser (405 nm) and Nanoparticle tracking Analysis (NTA) software (RRID:SCR_014239), version 2.3. Summary of exosome number and size distribution, with or without the finite track length adjustment (FTLA) algorithm, are provided in Table 1 as averaged values among the multiple independent exosome preparations generated during this study. Span was calculated using the formula: Span = (D90-D10)/D50, where D90 is the point in the size distribution where 90% of the sample is contained, D10 is where 10% of the sample is contained, and D50 is where 50% of the sample is contained (i.e. median). Batch analysis reports, including short video clips, are available at https://osf.io/vczn7/ for exosomes purified for in vitro experiments and https://osf.io/9xnbf/ for exosomes purified for in vivo experiments.

Western blots

Request a detailed protocol

Cell lysate was generated from B16-F10 shMet and B16-F10 shScr cells by removing medium from plates, washing 2X with 1X PBS, followed by addition of 1X RIPA buffer (Sigma-Aldrich, cat# R0278) supplemented with protease inhibitors (Sigma-Aldrich, cat# 4693116001) and phosphatase inhibitors (Sigma-Aldrich, cat# P5726) at manufacturer recommended concentrations. Lysed cells were scraped from plates, incubated on ice for 15 min followed by transfer to −80°C. Cell lysates were thawed on ice, then centrifuged at 12,000xg at 4°C before protein concentration of the supernatant was quantified using a BCA protein assay kit following manufacturer’s instructions. Previously frozen purified exosomes used for Western blots were resuspended in 1X RIPA buffer supplemented with protease and phosphatase inhibitors. Laemmli sample buffer was added to the purified exosomes or cell lysates and incubated at 95˚C for 5 min. 30 µg of purified exosome or cell lysates (or 10 µg of purified exosomes for gels examining exosome markers), along with a protein ladder (Bio-Rad Laboratories, cat# 161–0377), was resolved by SDS-PAGE and transferred to PVDF membrane as described in the Registered Report (Lesnik et al., 2016). Membranes were blocked with SuperBlock (Thermo Fisher Scientific, cat# 37516) following manufacturer’s instructions. Membranes were cut at ~75 kDa to allow for parallel probing. Incubation with primary antibody, diluted in SuperBlock, was conducted overnight at 4˚C. Membranes were probed with: mouse anti-Hsc70 (Enzo Life Sciences, cat# ALX-804–067, RRID:AB_10538284), 1:500 dilution; mouse anti-Tsg101 (Santa Cruz Biotechnology, cat# sc-7964, RRID:AB_671392), 1:500 dilution; rabbit anti-Cd63 (System Biosciences, cat# EXOAB-CD63A-1, RRID:AB_2561274), 1:1000 dilution; mouse anti-Met (Cell Signaling Technology, cat# 3127, RRID:AB_331361), 1:1000 dilution; rabbit anti-pMet (Tyr 1234/5) (Cell Signaling Technology, cat# 3077, RRID:AB_2315156), 1:1000 dilution; rabbit anti-Gapdh (Santa Cruz Biotechnology, cat# sc-25778, RRID:AB_10167668), 1:1000 dilution. Incubations were followed by washes with 1X TBS supplemented with 0.1% tween (TBST) and then the appropriate secondary antibody diluted in SuperBlock for 1 hr at room temperature: HRP-conjugated sheep anti-mouse (GE Healthcare, cat# NA931, RRID:AB_772210), 1:15,000 dilution; HRP-conjugated donkey anti-rabbit (GE Healthcare, cat# NA934, RRID:AB_772206), 1:15,000 dilution. Membranes were washed with TBST and incubated with SuperSignal West Femto Maximum Sensitivity Substrate (Thermo Fisher Scientific, cat# 34095) according to the manufacturer’s instructions. Scanned Western blots were quantified using ImageJ software (RRID:SCR_003070), version 1.49u (Schneider et al., 2012). We also used an additional antibody targeting pMet (rabbit anti-pMet (Tyr 1234/5) (Thermo Fisher Scientific, cat# MA5-15083, RRID:AB_10983015), 1:1000 dilution) due to an inability to detect a reliable Western blot signal in exosomes with the pMet antibody (RRID:AB_331361) stated in the Registered Report and used in the original study. Both antibodies gave similar results (RRID:AB_10983015: https://osf.io/96tcs/; RRID:AB_331361: Figure 1B). Additional method details and image data are available at https://osf.io/aqm2m/.

Quantitative PCR

Request a detailed protocol

Total RNA was isolated from B16-F10 shMet and B16-F10 shScr cells using TRIzol reagent (Thermo Fisher Scientific, cat# 15596026) according to manufacturer’s instructions. Total RNA was reverse transcribed into cDNA using a QuantiMir Kit (System Biosciences, cat# RA420A-1) according to manufacturer’s instructions. To detect the shMet shRNA expression a primer against the shMet sequence (Forward: 5’-CCAGACTTTTCATACAAGAATA-3’) and a universal reverse primer (QuantiMir Kit, System Biosciences, cat# RA420A-1) were used along with a U6 specific primer (LncRNA Profiler qPCR Array Kit, System Biosciences, cat# RA900A-1) and universal reverse primer that were used for normalization. To detect Met expression a Met specific primer (Forward: 5’-CTGGACAGATTGTGGGAGTAAG −3’) and universal reverse primer were used along with a Gapdh specific primer (LncRNA Profiler qPCR Array Kit) and universal reverse primer that were used for normalization. qRT-PCR reactions were performed in technical triplicate with Power SYBR Green PCR Master Mix (Thermo Fisher Scientific, cat# 4368577) according to manufacturer’s instructions. PCR cycling conditions were [1 cycle 50°C for 2 min, 95°C for 10 min – 40 cycles 95°C for 15 s, 60°C for 60 s – dissociation stage: 95°C for 15 s, 60°C for 15 s] using an Applied Biosystems real-time PCR system (Applied Biosystems, 7900 HT Fast Real-Time PCR System) and SDS software (RRID:SCR_015806), version 2.4. Negative controls containing no cDNA template were included. Relative expression levels were determined using the ∆∆Ct method.

Animals

All animal procedures were approved by the Stanford University IACUC# 30226 and were in accordance with the Stanford University policies on the care, welfare, and treatment of laboratory animals. No blinding occurred during the experiments.

Six-week old female C57BL/6 mice (Charles River, Strain code: 027, RRID:IMSR_CRL:27) were housed in sterile conditions under standard temperature, humidity, and timed lighting conditions with 12 hr light/dark cycles, and were provided with sterile rodent chow and water ad libitum. Mice were randomized and injected, via retro-orbital and alternating injections between left and right eyes, three times a week for a total of 28 days with either 5 µg of freshly-isolated B16-F10 shScr exosomes, 5 µg of freshly-isolated B16-F10 shMet exosomes, or 1.25 µg synthetic 100 nm liposomes (Encapsula NanoSciences) (mimics 5 µg exosome protein, based on a theoretical 4:1 protein:L-α-phosphatidylcholine ratio) in 100 µl filtered Phosphate Buffered Saline (PBS) (Thermo Fisher Scientific, cat# MT21040CM). The exosomes were resuspended right before each individual injection by pipetting 3–4 times, tapping the tube, inverted 3–4 times, and then tapping the syringe right before each injection. After exosome injections (12 total), mice were inoculated subcutaneously (s.c.) with 1×10⁶ B16-F10-luciferase cells in 100 µl of PBS in the dorsal area. Primary tumor (caliper measurements) and body weight were measured three times a week. The planned study design involved waiting 21 days after B16-F10-luciferase tumor cell implantation before sacrificing the mice for analysis; however, two animals were found dead (one injected with B16-F19 shScr exosomes and one injected with synthetic liposomes) before this time point was reached (17 days after implantation), which in addition the largest tumors of the surviving mice having reached >1000 mm³, prompted us to stop the experiment early (18 days after implantation). To measure metastasis, mice were anesthetized (using isoflurane and O₂) and 50 mg/kg of D-luciferin in 50–100 µl PBS was injected retro-orbitally. Five minutes later, mice were euthanized by cervical dislocation and primary tumors and organs (femurs and lungs) were dissected. Primary tumors were weighed (Delta Range scale: Metler Toledo, Model # PB303-S) and organs (femurs and lungs) were analyzed for luciferase expression using the IVIS Spectrum system (Caliper, Xenogen). Anesthesia, luciferin injections, euthanasia, dissection, and imaging/weighing were performed with mice from different groups in parallel (i.e. one from each of the three groups) so variations during the procedure were equal across groups.

IVIS imaging

Request a detailed protocol

Images were acquired in a Xenogen IVIS Spectrum at a medium binning level (8) and a 22.6 cm field of view. Acquisition times were set to auto-exposure and were required at 1 and 4 min. There was high concordance between the two exposures (correlation coefficient (ρ)=0.97). The 4 min exposure is shown in figures and used in the statistical analysis. Additionally, after this imaging was complete, the organs were soaked for 10–15 min in 4 ml of 50 mg/ml D-luciferin in six well tissue culture plates and imaged again at 1 and 3 min, which had high concordance with the first set of images taken. Living Image software (RRID:SCR_014247), version 4.4, was used for quantitative analysis. Image files are available at https://osf.io/9xnbf/.

Statistical analysis

Request a detailed protocol

Statistical analysis was performed with R software (RRID:SCR_001905), version 3.5.1 (Core TeamR, 2018). All data, csv files, and analysis scripts are available on the OSF (https://osf.io/ewqzf/). Confirmatory statistical analysis was pre-registered (https://osf.io/g5tzn/) before the experimental work began as outlined in the Registered Report (Lesnik et al., 2016). Data were checked to ensure assumptions of statistical tests were met and in the case of the metastatic data were natural log transformed to achieve a normal distribution and equal variance while also allowing for comparisons on a modified percentage scale (Cole and Altman, 2017). When described in the results, the Bonferroni correction, to account for multiple testings, was applied to the alpha error or the p-value. The Bonferroni corrected value was determined by divided the uncorrected value (0.05) by the number of tests performed. A meta-analysis of a common original and replication effect size was performed with a random effects model and the metafor R package (Viechtbauer, 2010) (https://osf.io/c69jx/). Meta-analyses were performed without weighting, since unweighted Cliff’s d has been reported to reduce bias (Kromrey et al., 2005). The asymmetric confidence intervals for the overall Cliff’s d estimate was determined using the normal deviate corresponding to the (1 - alpha/2)^th percentile of the normal distribution (Cliff, 1993). The raw data pertaining to Figure 4E and Supplementary Figure 5A of Peinado et al., 2012 were shared by the original authors. The summary data was published in the Registered Report (Lesnik et al., 2016) and used in the power calculations to determine the sample size for this study.

Data availability

Request a detailed protocol

Additional detailed experimental notes, data, and analysis are available on OSF (RRID:SCR_003238) (https://osf.io/ewqzf/; Kim et al., 2018). This includes the R Markdown file (https://osf.io/hz3k7/) that was used to compose this manuscript, which is a reproducible document linking the results in the article directly to the data and code that produced them (Hartgerink, 2017).

Deviations from registered report

Request a detailed protocol

We planned to generated B16-F10 shMet and shScr stable cells with an MOI of 10, similar to the original study, but observed that the Met levels in the shScr cells at this MOI were, for unknown reasons, low when compared to the shScr cells generated at the other MOI ratios (Figure 1—figure supplement 1C). Experiments reported in this study were with stable cells generated with an MOI of 20. We also included additional tests (qRT-PCR) to confirm the presence of shMet, and corresponding decrease in Met expression, in the stable cells generated. The cell lysis buffer was additionally supplemented with phosphatase inhibitors in an attempt to increase detection of pMet. We also tried an additional antibody targeted against pMet (RRID:AB_10983015) due to an inability to detect a reliable Western blot signal in exosomes with the p-Met antibody stated in the Registered Report and used in the original study (RRID:AB_2315156). Both antibodies gave similar results (RRID:AB_10983015: https://osf.io/96tcs/; RRID:AB_331361: Figure 1B). For the in vivo experimentation, the planned study design involved waiting 21 days after B16-F10-luciferase tumor cell implantation before sacrificing the mice for analysis; however, two animals were found dead (one injected with B16-F19 shScr exosomes and one injected with synthetic liposomes) before this time point was reached, which in addition the largest tumors of the surviving mice having reached >1000 mm³, prompted us to stop the experiment early (18 days after implantation). Additional materials and instrumentation not listed in the Registered Report, but needed during experimentation are also listed.

Data availability

The following data sets were generated

1. Kim J
2. Afshari A
3. Sengupta R
4. Sebastiano V
5. Gupta A
6. Kim YH
7. Iorns E
8. Tsui R
9. Denis A
10. Perfito N
11. Errington TM
(2018) Open Science Framework
Study 42: Replication of Peinado et al., 2012 (Nature Medicine).

https://doi.org/10.17605/OSF.IO/EWQZF

References

1. Adachi E
2. Sakai K
3. Nishiuchi T
4. Imamura R
5. Sato H
6. Matsumoto K
(2016) Different growth and metastatic phenotypes associated with a cell-intrinsic change of Met in metastatic melanoma
Oncotarget 7:70779–70793.

https://doi.org/10.18632/oncotarget.12221
- PubMed
- Google Scholar
(2014) Refinement of experimental design and conduct in laboratory animal research
ILAR Journal 55:383–391.

https://doi.org/10.1093/ilar/ilu037
- PubMed
- Google Scholar
1. Ben-David U
2. Siranosian B
3. Ha G
4. Tang H
5. Oren Y
6. Hinohara K
7. Strathdee CA
8. Dempster J
9. Lyons NJ
10. Burns R
11. Nag A
12. Kugener G
13. Cimini B
14. Tsvetkov P
15. Maruvka YE
16. O'Rourke R
17. Garrity A
18. Tubelli AA
19. Bandopadhayay P
20. Tsherniak A
21. Vazquez F
22. Wong B
23. Birger C
24. Ghandi M
25. Thorner AR
26. Bittker JA
27. Meyerson M
28. Getz G
29. Beroukhim R
30. Golub TR
(2018) Genetic and transcriptional evolution alters cancer cell line drug response
Nature 560:325–330.

https://doi.org/10.1038/s41586-018-0409-3
- PubMed
- Google Scholar
1. Cannistraci A
2. Federici G
3. Addario A
4. Di Pace AL
5. Grassi L
6. Muto G
7. Collura D
8. Signore M
9. De Salvo L
10. Sentinelli S
11. Simone G
12. Costantini M
13. Nanni S
14. Farsetti A
15. Coppola V
16. De Maria R
17. Bonci D
(2017) C-Met/miR-130b axis as novel mechanism and biomarker for castration resistance state acquisition
Oncogene 36:3718–3728.

https://doi.org/10.1038/onc.2016.505
- PubMed
- Google Scholar
1. Casellas J
(2011) Inbred mouse strains and genetic stability: a review
Animal 5:1–7.

https://doi.org/10.1017/S1751731110001667
- PubMed
- Google Scholar
1. Chen IH
2. Xue L
3. Hsu CC
4. Paez JS
5. Pan L
6. Andaluz H
7. Wendt MK
8. Iliuk AB
9. Zhu JK
10. Tao WA
(2017) Phosphoproteins in extracellular vesicles as candidate markers for breast cancer
PNAS 114:3175–3180.

https://doi.org/10.1073/pnas.1618088114
- PubMed
- Google Scholar
1. Cliff N
(1993) Dominance statistics: Ordinal analyses to answer ordinal questions
Psychological Bulletin 114:494–509.

https://doi.org/10.1037/0033-2909.114.3.494
- Google Scholar
1. Cole TJ
2. Altman DG
(2017) Statistics Notes: Percentage differences, symmetry, and natural logarithms
Bmj 358:j3683.

https://doi.org/10.1136/bmj.j3683
- PubMed
- Google Scholar
Software
1. Core TeamR
(2018)
R: A Language and Environment for Statistical Computing

R Foundation for Statistical Computing, Austria.
1. Cornelis F
2. Saut O
3. Cumsille P
4. Lombardi D
5. Iollo A
6. Palussiere J
7. Colin T
(2013) In vivo mathematical modeling of tumor growth from imaging data: soon to come in the future?
Diagnostic and Interventional Imaging 94:593–600.

https://doi.org/10.1016/j.diii.2013.03.001
- PubMed
- Google Scholar
1. Costa-Silva B
2. Aiello NM
3. Ocean AJ
4. Singh S
5. Zhang H
6. Thakur BK
7. Becker A
8. Hoshino A
9. Mark MT
10. Molina H
11. Xiang J
12. Zhang T
13. Theilen TM
14. García-Santos G
15. Williams C
16. Ararso Y
17. Huang Y
18. Rodrigues G
19. Shen TL
20. Labori KJ
21. Lothe IM
22. Kure EH
23. Hernandez J
24. Doussot A
25. Ebbesen SH
26. Grandgenett PM
27. Hollingsworth MA
28. Jain M
29. Mallya K
30. Batra SK
31. Jarnagin WR
32. Schwartz RE
33. Matei I
34. Peinado H
35. Stanger BZ
36. Bromberg J
37. Lyden D
(2015) Pancreatic cancer exosomes initiate pre-metastatic niche formation in the liver
Nature Cell Biology 17:816–826.

https://doi.org/10.1038/ncb3169
- PubMed
- Google Scholar
(1996) Two-Sided P-Values from Discrete Asymmetric Distributions Based on Uniformly Most Powerful Unbiased Tests
The Statistician 45:397.

https://doi.org/10.2307/2988542
- Google Scholar
1. Errington TM
2. Iorns E
3. Gunn W
4. Tan FE
5. Lomax J
6. Nosek BA
(2014) An open investigation of the reproducibility of cancer biology research
eLife 3:04333.

https://doi.org/10.7554/eLife.04333
- Google Scholar
1. Fong MY
2. Zhou W
3. Liu L
4. Alontaga AY
5. Chandra M
6. Ashby J
7. Chow A
8. O'Connor ST
9. Li S
10. Chin AR
11. Somlo G
12. Palomares M
13. Li Z
14. Tremblay JR
15. Tsuyada A
16. Sun G
17. Reid MA
18. Wu X
19. Swiderski P
20. Ren X
21. Shi Y
22. Kong M
23. Zhong W
24. Chen Y
25. Wang SE
(2015) Breast-cancer-secreted miR-122 reprograms glucose metabolism in premetastatic niche to promote metastasis
Nature Cell Biology 17:183–194.

https://doi.org/10.1038/ncb3094
- PubMed
- Google Scholar
1. Guo R
2. Lee Y
3. Byrnes C
4. Miller J
(2017) Puromycin Selection Confounds the RNA-Seq Profiles of Primary Human Erythroblasts
Transcriptomics: Open Access 5:.

https://doi.org/10.4172/2329-8936.1000140
- Google Scholar
Website
1. Hartgerink CHJ
(2017) Composing reproducible manuscripts using R Markdown
eLife. Accessed November 20, 2017.

https://elifesciences.org/labs/cad57bcf/composing-reproducible-manuscripts-using-r-markdown
1. He M
2. Qin H
3. Poon TC
4. Sze SC
5. Ding X
6. Co NN
7. Ngai SM
8. Chan TF
9. Wong N
(2015) Hepatocellular carcinoma-derived exosomes promote motility of immortalized hepatocyte through transfer of oncogenic proteins and RNAs
Carcinogenesis 36:1008–1018.

https://doi.org/10.1093/carcin/bgv081
- PubMed
- Google Scholar
1. Hoshino A
2. Costa-Silva B
3. Shen TL
4. Rodrigues G
5. Hashimoto A
6. Tesic Mark M
7. Molina H
8. Kohsaka S
9. Di Giannatale A
10. Ceder S
11. Singh S
12. Williams C
13. Soplop N
14. Uryu K
15. Pharmer L
16. King T
17. Bojmar L
18. Davies AE
19. Ararso Y
20. Zhang T
21. Zhang H
22. Hernandez J
23. Weiss JM
24. Dumont-Cole VD
25. Kramer K
26. Wexler LH
27. Narendran A
28. Schwartz GK
29. Healey JH
30. Sandstrom P
31. Labori KJ
32. Kure EH
33. Grandgenett PM
34. Hollingsworth MA
35. de Sousa M
36. Kaur S
37. Jain M
38. Mallya K
39. Batra SK
40. Jarnagin WR
41. Brady MS
42. Fodstad O
43. Muller V
44. Pantel K
45. Minn AJ
46. Bissell MJ
47. Garcia BA
48. Kang Y
49. Rajasekhar VK
50. Ghajar CM
51. Matei I
52. Peinado H
53. Bromberg J
54. Lyden D
(2015) Tumour exosome integrins determine organotropic metastasis
Nature 527:329–335.

https://doi.org/10.1038/nature15756
- PubMed
- Google Scholar
1. Howard BR
(2002) Control of variability
ILAR Journal 43:194–201.

https://doi.org/10.1093/ilar.43.4.194
- PubMed
- Google Scholar
1. Hughes P
2. Marshall D
3. Reid Y
4. Parkes H
5. Gelber C
(2007) The costs of using unauthenticated, over-passaged cell lines: how much more data do we need?
BioTechniques 43:575–586.

https://doi.org/10.2144/000112598
- PubMed
- Google Scholar
1. Jensen MN
2. Ritskes-Hoitinga M
(2007) How isoflavone levels in common rodent diets can interfere with the value of animal models and with experimental results
Laboratory Animals 41:1–18.

https://doi.org/10.1258/002367707779399428
- PubMed
- Google Scholar
1. Keerthikumar S
2. Gangoda L
3. Liem M
4. Fonseka P
5. Atukorala I
6. Ozcitti C
7. Mechler A
8. Adda CG
9. Ang CS
10. Mathivanan S
(2015) Proteogenomic analysis reveals exosomes are more oncogenic than ectosomes
Oncotarget 6:15375–15396.

https://doi.org/10.18632/oncotarget.3801
- PubMed
- Google Scholar
1. Keerthikumar S
2. Chisanga D
3. Ariyaratne D
4. Al Saffar H
5. Anand S
6. Zhao K
7. Samuel M
8. Pathan M
9. Jois M
10. Chilamkurti N
11. Gangoda L
12. Mathivanan S
(2016) ExoCarta: A Web-Based Compendium of Exosomal Cargo
Journal of Molecular Biology 428:688–692.

https://doi.org/10.1016/j.jmb.2015.09.019
- PubMed
- Google Scholar
1. Kim J
2. Afshari A
3. Sengupta R
4. Sebastiano V
5. Gupta A
6. Kim YH
7. Iorns E
8. Tsui R
9. Denis A
10. Perfito N
11. Errington TM
(2018) Study 42: Replication of Peinado et al., 2012 (Nature Medicine)
Open Science Framework.

https://doi.org/10.17605/OSF.IO/EWQZF
- Google Scholar
1. Kleensang A
2. Vantangoli MM
3. Odwin-DaCosta S
4. Andersen ME
5. Boekelheide K
6. Bouhifd M
7. Fornace AJ
8. Li HH
9. Livi CB
10. Madnick S
11. Maertens A
12. Rosenberg M
13. Yager JD
14. Zhao L
15. Hartung T
(2016) Genetic variability in a frozen batch of MCF-7 cells invisible in routine authentication affecting cell function
Scientific Reports 6:28994.

https://doi.org/10.1038/srep28994
- PubMed
- Google Scholar
1. Kokolus KM
2. Capitano ML
3. Lee CT
4. Eng JW
5. Waight JD
6. Hylander BL
7. Sexton S
8. Hong CC
9. Gordon CJ
10. Abrams SI
11. Repasky EA
(2013) Baseline tumor growth and immune control in laboratory mice are significantly influenced by subthermoneutral housing temperature
PNAS 110:20176–20181.

https://doi.org/10.1073/pnas.1304291110
- PubMed
- Google Scholar
1. Kromrey JD
2. Hogarty KY
3. Ferron JM
4. Hines CV
5. Hess MR
(2005)
Robustness in meta-analysis: An empirical comparison of point and interval estimates of standardized mean differences and Cliff’s delta

Minneapolis .
- Google Scholar
1. Lazar I
2. Clement E
3. Ducoux-Petit M
4. Denat L
5. Soldan V
6. Dauvillier S
7. Balor S
8. Burlet-Schiltz O
9. Larue L
10. Muller C
11. Nieto L
(2015) Proteome characterization of melanoma exosomes reveals a specific signature for metastatic cell lines
Pigment Cell & Melanoma Research 28:464–475.

https://doi.org/10.1111/pcmr.12380
- PubMed
- Google Scholar
1. Leek JT
2. Scharpf RB
3. Bravo HC
4. Simcha D
5. Langmead B
6. Johnson WE
7. Geman D
8. Baggerly K
9. Irizarry RA
(2010) Tackling the widespread and critical impact of batch effects in high-throughput data
Nature Reviews Genetics 11:733–739.

https://doi.org/10.1038/nrg2825
- PubMed
- Google Scholar
(2016) Registered report: Melanoma exosomes educate bone marrow progenitor cells toward a pro-metastatic phenotype through MET
eLife 5:e07383.

https://doi.org/10.7554/eLife.07383
- PubMed
- Google Scholar
1. Li J
2. Chen X
3. Yi J
4. Liu Y
5. Li D
6. Wang J
7. Hou D
8. Jiang X
9. Zhang J
10. Wang J
11. Zen K
12. Yang F
13. Zhang CY
14. Zhang Y
(2016) Identification and Characterization of 293T Cell-Derived Exosomes by Profiling the Protein, mRNA and MicroRNA Components
PLOS ONE 11:e0163043.

https://doi.org/10.1371/journal.pone.0163043
- PubMed
- Google Scholar
1. Liang B
2. Peng P
3. Chen S
4. Li L
5. Zhang M
6. Cao D
7. Yang J
8. Li H
9. Gui T
10. Li X
11. Shen K
(2013) Characterization and proteomic analysis of ovarian cancer-derived exosomes
Journal of Proteomics 80:171–182.

https://doi.org/10.1016/j.jprot.2012.12.029
- PubMed
- Google Scholar
1. Liu Y
2. Gu Y
3. Han Y
4. Zhang Q
5. Jiang Z
6. Zhang X
7. Huang B
8. Xu X
9. Zheng J
10. Cao X
(2016) Tumor Exosomal RNAs Promote Lung Pre-metastatic Niche Formation by Activating Alveolar Epithelial TLR3 to Recruit Neutrophils
Cancer Cell 30:243–256.

https://doi.org/10.1016/j.ccell.2016.06.021
- PubMed
- Google Scholar
1. Macpherson AJ
2. McCoy KD
(2015) Standardised animal models of host microbial mutualism
Mucosal Immunology 8:476–486.

https://doi.org/10.1038/mi.2014.113
- PubMed
- Google Scholar
1. Nevalainen T
(2014) Animal husbandry and experimental design
ILAR Journal 55:392–398.

https://doi.org/10.1093/ilar/ilu035
- PubMed
- Google Scholar
1. Peinado H
2. Alečković M
3. Lavotshkin S
4. Matei I
5. Costa-Silva B
6. Moreno-Bueno G
7. Hergueta-Redondo M
8. Williams C
9. García-Santos G
10. Ghajar C
11. Nitadori-Hoshino A
12. Hoffman C
13. Badal K
14. Garcia BA
15. Callahan MK
16. Yuan J
17. Martins VR
18. Skog J
19. Kaplan RN
20. Brady MS
21. Wolchok JD
22. Chapman PB
23. Kang Y
24. Bromberg J
25. Lyden D
(2012) Melanoma exosomes educate bone marrow progenitor cells toward a pro-metastatic phenotype through MET
Nature Medicine 18:883–891.

https://doi.org/10.1038/nm.2753
- PubMed
- Google Scholar
1. Plebanek MP
2. Angeloni NL
3. Vinokour E
4. Li J
5. Henkin A
6. Martinez-Marin D
7. Filleur S
8. Bhowmick R
9. Henkin J
10. Miller SD
11. Ifergan I
12. Lee Y
13. Osman I
14. Thaxton CS
15. Volpert OV
(2017) Pre-metastatic cancer exosomes induce immune surveillance by patrolling monocytes at the metastatic niche
Nature Communications 8:1319.

https://doi.org/10.1038/s41467-017-01433-3
- PubMed
- Google Scholar
1. Rohatgi VK
2. Saleh A
(2000) An Introduction to Probability and Statistics: Rohatgi/An Introduction, Wiley Series in Probability and Statistics
Wiley Online Library.

https://doi.org/10.1002/9781118165676
- Google Scholar
(2012) NIH Image to ImageJ: 25 years of image analysis
Nature Methods 9:671–675.

https://doi.org/10.1038/nmeth.2089
- PubMed
- Google Scholar
1. Shearer RF
2. Saunders DN
(2015) Experimental design for stable genetic manipulation in mammalian cell lines: lentivirus and alternatives
Genes to Cells 20:1–10.

https://doi.org/10.1111/gtc.12183
- PubMed
- Google Scholar
1. Sorge RE
2. Martin LJ
3. Isbester KA
4. Sotocinal SG
5. Rosen S
6. Tuttle AH
7. Wieskopf JS
8. Acland EL
9. Dokova A
10. Kadoura B
11. Leger P
12. Mapplebeck JC
13. McPhail M
14. Delaney A
15. Wigerblad G
16. Schumann AP
17. Quinn T
18. Frasnelli J
19. Svensson CI
20. Sternberg WF
21. Mogil JS
(2014) Olfactory exposure to males, including men, causes stress and related analgesia in rodents
Nature Methods 11:629–632.

https://doi.org/10.1038/nmeth.2935
- PubMed
- Google Scholar
1. Steenbeek SC
2. Pham TV
3. de Ligt J
4. Zomer A
5. Knol JC
6. Piersma SR
7. Schelfhorst T
8. Huisjes R
9. Schiffelers RM
10. Cuppen E
11. Jimenez CR
12. van Rheenen J
(2018) Cancer cells copy migratory behavior and exchange signaling networks via extracellular vesicles
The EMBO Journal 37:e98357.

https://doi.org/10.15252/embj.201798357
- PubMed
- Google Scholar
1. Talkington A
2. Durrett R
(2015) Estimating Tumor Growth Rates In Vivo
Bulletin of Mathematical Biology 77:1934–1954.

https://doi.org/10.1007/s11538-015-0110-8
- PubMed
- Google Scholar
1. Valentine JC
2. Biglan A
3. Boruch RF
4. Castro FG
5. Collins LM
6. Flay BR
7. Kellam S
8. Mościcki EK
9. Schinke SP
(2011) Replication in prevention science
Prevention Science 12:103–117.

https://doi.org/10.1007/s11121-011-0217-6
- PubMed
- Google Scholar
1. Viechtbauer W
(2010) Conducting Meta-Analyses inRwith themetaforPackage
Journal of Statistical Software 36:.

https://doi.org/10.18637/jss.v036.i03
- Google Scholar
1. Zeng AL
2. Yan W
3. Liu YW
4. Wang Z
5. Hu Q
6. Nie E
7. Zhou X
8. Li R
9. Wang XF
10. Jiang T
11. You YP
(2017) Tumour exosomes from cells harbouring PTPRZ1-MET fusion contribute to a malignant phenotype and temozolomide chemoresistance in glioblastoma
Oncogene 36:5369–5381.

https://doi.org/10.1038/onc.2017.134
- PubMed
- Google Scholar
1. Zhou W
2. Fong MY
3. Min Y
4. Somlo G
5. Liu L
6. Palomares MR
7. Yu Y
8. Chow A
9. O'Connor ST
10. Chin AR
11. Yen Y
12. Wang Y
13. Marcusson EG
14. Chu P
15. Wu J
16. Wu X
17. Li AX
18. Li Z
19. Gao H
20. Ren X
21. Boldin MP
22. Lin PC
23. Wang SE
(2014) Cancer-secreted miR-105 destroys vascular endothelial barriers to promote metastasis
Cancer Cell 25:501–515.

https://doi.org/10.1016/j.ccr.2014.03.007
- PubMed
- Google Scholar

Article and author information

Author details

Jeewon Kim

Stanford Transgenic, Knockout and Tumor Model Center, Stanford Cancer Institute, California, United States

Contribution
Acquisition of data, Analysis and interpretation of data, Drafting or revising the article, Performed all exosome purification and generation of exosome-free FBS, qPCR, Animal experiments and IVIS imaging

Competing interests
Stanford Transgenic, Knockout and Tumor model Center is a Science Exchange associated lab.
Amirali Afshari

System Biosciences LLC, California, United States

Contribution
Acquisition of data, Analysis and interpretation of data, Drafting or revising the article, Performed stable cell line generation, Western blots, qPCR, Nanosight analysis

Contributed equally with
Ranjita Sengupta

Competing interests
System Biosciences LLC is a Science Exchange associated lab and offers products and services for exosome research. Exosome detection products, specifically exosome specific antibodies, and nanoparticle tracking analysis (NTA) service were used during this study; however, ultracentrifugation-free exosome isolation products and services were not used.
Ranjita Sengupta

System Biosciences LLC, California, United States

Contribution
Acquisition of data, Analysis and interpretation of data, Drafting or revising the article, Performed stable cell line generation, Western blots, qPCR, Nanosight analysis

Contributed equally with
Amirali Afshari

Competing interests
System Biosciences LLC is a Science Exchange associated lab and offers products and services for exosome research. Exosome detection products, specifically exosome specific antibodies, and nanoparticle tracking analysis (NTA) service were used during this study; however, ultracentrifugation-free exosome isolation products and services were not used.
Vittorio Sebastiano
1. Stanford Transgenic, Knockout and Tumor Model Center, Stanford Cancer Institute, California, United States
2. The Institute for Stem Cell Biology and Regenerative Medicine, Stanford, United States
3. Department of Obstetrics and Gynecology, Stanford School of Medicine, Stanford, United States
Contribution
Analysis and interpretation of data, Drafting or revising the article

Competing interests
Stanford Transgenic, Knockout and Tumor model Center is a Science Exchange associated lab.
Archana Gupta

System Biosciences LLC, California, United States

Contribution
Acquisition of data, Analysis and interpretation of data, Drafting or revising the article, Performed stable cell line generation, Western blots, qPCR, Nanosight analysis

Competing interests
System Biosciences LLC is a Science Exchange associated lab and offers products and services for exosome research. Exosome detection products, specifically exosome specific antibodies, and nanoparticle tracking analysis (NTA) service were used during this study; however, ultracentrifugation-free exosome isolation products and services were not used.
Young H Kim

System Biosciences LLC, California, United States

Contribution
Acquisition of data, Analysis and interpretation of data, Drafting or revising the article, Performed stable cell line generation, Western blots, qPCR, Nanosight analysis

Competing interests
System Biosciences LLC is a Science Exchange associated lab and offers products and services for exosome research. Exosome detection products, specifically exosome specific antibodies, and nanoparticle tracking analysis (NTA) service were used during this study; however, ultracentrifugation-free exosome isolation products and services were not used.
Reproducibility Project: Cancer Biology

Contribution
Analysis and interpretation of data, Drafting or revising the article
For correspondence
1. tim@cos.io
2. nicole@scienceexchange.com
Competing interests
EI, RT, NP: Employed by and hold shares in Science Exchange Inc
1. Elizabeth Iorns, Science Exchange, Palo Alto, United States
2. Rachel Tsui, Science Exchange, Palo Alto, United States
3. Alexandria Denis, Center for Open Science, Charlottesville, United States
4. Nicole Perfito, Science Exchange, Palo Alto, United States
5. Timothy M Errington, Center for Open Science, Charlottesville, United States
6. Elizabeth Iorns, Science Exchange, Palo Alto, United States
7. Rachel Tsui, Science Exchange, Palo Alto, United States
8. Alexandria Denis, Center for Open Science, Charlottesville, United States
9. Nicole Perfito, Science Exchange, Palo Alto, United States
10. Timothy M Errington, Center for Open Science, Charlottesville, United States

Funding

Laura and John Arnold Foundation

Reproducibility Project: Cancer Biology

The funder had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology would like to thank Dr. David Lyden (Weill Medical College of Cornell University) and Hector Peinado (Spanish National Cancer Research Centre) for sharing critical reagents, data, and protocol information during preparation of the Registered Report, specifically the B16-F10 and B16-F10-luciferase cells. We would also like to thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses, Jacob Lesnik for managing resources and collaboration details at System Biosciences LLC, the Stanford Transgenic, Knockout and Tumor model Center at the Stanford Cancer Institute, and the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type and Tissue Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, and System Biosciences (SBI).

Ethics

Animal experimentation: All animal procedures were approved by the Stanford University IACUC# 30226 and were in accordance with the Stanford University policies on the care, welfare, and treatment of laboratory animals.

Copyright

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.