Introduction

Meta-analyses, or quantitative syntheses of results from different studies (Koricheva and Gurevitch 2014), have a long history in archaeology in general and in archaeozoology in particular (Jones in review). The increasing publication of archaeozoological datasets (e.g., Arbuckle et al. 2014; Cahill et al. 2014) means zooarchaeological meta-analyses are ever-more feasible, and this opportunity has been realized in a suite of new and exciting studies (for example, Jones 2015; Manning et al. 2013; Nelson and Schollmeyer 2003; Orton et al. 2014; Ottoni et al. 2013; Shennan et al. 2013). However, the increasing ease of zooarchaeological meta-analysis also comes with hazards, and one of these is the inclusion of older collections and datasets in analysis.

Meta-analyses will always contain problematic assemblages; the whole point of such studies is to include as many data points as possible (Koricheva and Gurevitch 2014). By including many datasets, the reasoning is, error in individual datasets should simply appear as ‘noise’ in the overall analysis. In most cases, dataset-specific problems will be offset by other datasets. In many studies, this approach works well. However, when there is a systematic bias in assemblages – whether a site type bias (Jones 2013a), analyst-driven bias (Gobalet 2001), or collection methodology bias (Jones in review) – if care is not taken to address these errors, meta-analyses can magnify these problems rather than reducing them.

Older collections and datasets are one potential source of such bias. In our definition, older collections may (1) be disassociated from their initial site or project; (2) have gone through not just post-depositional processes before their excavation, but post-collection processes as well (potentially including the culling or trading of material, either for analytical purposes or because the material was thought insignificant); (3) no longer be housed together; (4) vary in state of preservation and curation; and/or (5) lack provenience and collection information. These characteristics are often found in collections excavated and curated over 30 years ago. It is important to note, however, that more recent collections sometimes share these traits, and some chronologically older collections do not. In short, in our definition “older collections” are characterized by condition rather than by age.

In regions with a long excavation history but a relatively minimal tradition of archaeozoological research, such as the American Southwest (Gifford-Gonzalez 2011), meta-analyses will inevitably include a combination of older and newer collections and datasets, thus raising issues as to the comparability of those assemblages. Older collections may differ significantly in their composition from newer ones as a result of differences in archaeological collection and curation rather than differences in the behavior of the humans who initially deposited the fauna. In this paper, we use examples from our research on late prehistoric and early historic faunas of New Mexico and a case study on the distribution of Old World domestic fauna in 17th century New Mexico to explore issues raised by the inclusion of older collections in archaeozoological meta-analyses.

Older collections: issues and impacts

While every archaeozoological collection’s life history is, of course, unique, there are several factors commonly encountered when working with older collections. Here we discuss three of them – variability in screen size, contextual biases, and documentation (or lack thereof) – and then explore when these factors will likely impact results in meta-analyses.

Screening (or lack thereof)

As has been widely demonstrated, screening protocol during excavation has a profound impact on the composition of zooarchaeological assemblages (Cannon 1999; Nagaoka 2005; Schaffer 1992; Schaffer and Sanchez 1994; Stahl 1996). If not taken into account, variability in screening can easily cause patterns that appear to represent changes in human subsistence behavior, but in fact reflect changes in archaeological practice (Grayson and Cannon 1999; Jones 2013b).

Understanding what type of screening was used in excavation is thus critical for any analysis involving older collections. Screening was rare in excavations prior to the late 1960s (see discussion in James 1997). In the American Southwest, the use of quarter-inch mesh became widespread (although not ubiquitous) in the 1970s, and as the influence of screen size on recovery became more well-known, eighth-inch screens were increasingly used in some (but usually not all) contexts from the late 1980s on (for example, Hovezak, Sesler and Fuller 2002).

This variability in screening practice is a source of significant challenges for the archaeozoologist (Jones 2013b), and such challenges are especially profound in meta-analyses where the data included may range from collections for which screening protocols are unknown to collections derived from a combination of flotation, screening with quarter-inch mesh, and no screening at all. In such situations, understanding the factors driving taxonomic relative abundances is exceptionally difficult. Our work with Tijeras Pueblo (LA 581) provides an example of this. Tijeras Pueblo is an archaeological site dating approximately between A.D. 1300 and 1425 (Cordell 1980), located in the Sandia Mountains to the east of modern-day Albuquerque (Figure 1). The Tijeras fauna are preserved in at least three separate collections produced by three different excavation episodes: a UNM field school in 1948 (directed by Stanley Stubbs); salvage excavation in 1968 and 1969 (directed by Stewart Peckham); and another UNM field school from 1971–1978 (directed by W. James Judge 1971–73 and Linda Cordell 1974–76) (Cordell 1977; Judge 1974). Because the 1948 field school faunal collections discovered thus far are limited in number (NSP < 50), it is not possible to use these fauna to assess collection method. The 1968/1969 salvage and 1970s field school collections, however, are large enough to support further analysis.

Figure 1 

Map of the study area.

The collections from the 1968/1969 Tijeras Pueblo salvage excavations are housed at the Museum of Indian Arts and Culture in Santa Fe, New Mexico, and while some field notes are available, relatively little is known about the methodologies employed in this project. Even the location of the excavations is a matter of dispute (Cordell 1977; Judge 1974). The 1971–78 collections housed at the Maxwell Museum of Anthropology in Albuquerque, on the other hand, are reasonably well-documented (Cordell 1980; Young 1980). We know, for instance, that the UNM field school used quarter-inch mesh, while we have no information about screening protocols in 1968. A comparison of taxonomic abundances between the two collections suggests screening was limited at best in the earlier excavation (Figure 2). There is a significant difference (χ2 = 269.550, p = .000) between the two collections, with large mammals over-represented and smaller taxa (lagomorphs and birds) under-represented in the fauna recovered in 1968/1969. This pattern is what would be expected in an un-screened assemblage.

Figure 2 

Taxonomic relative abundance at Tijeras Pueblo by excavation episode.

This analysis is not, of course, a meta-analysis. The Tijeras Pueblo faunal data derives from a single archaeological site, albeit one with several collection episodes. We also do have some documentation of these collections; for instance, the fact that the 1968 and 1970s collections are housed separately means that we are able to compare and draw conclusions about the impacts of screening protocols. This is not always the case even within a single set of assemblages, let alone in a meta-analysis containing hundreds of data points. In a meta-analysis comprising data from both unscreened and screened archaeofaunal assemblages, patterns will emerge that have nothing to do with past human behavior or even environmental change, but rather with changing archaeological method. If differences in screening protocol align with other variables of interest – for instance, if high-elevation sites were excavated earlier and were not screened, while low-elevation ones were investigated more recently and were screened – spurious correlations will result.

Context

Screening is not the only methodological choice that can produce misleading patterns in taxonomic relative abundance. Differences in site type and function can also impact meta-analyses in this way (Jones 2013a; Straus 2006). At Tijeras Pueblo, a problem analogous to that of site type can be observed when fauna are considered by spatial context. Like other Ancient Puebloan masonry structures from the Southwest, Tijeras Pueblo has several distinct contexts: rooms (within the main structure(s) of the site), plazas (open-air areas associated with the roomblocks), and middens (buried trash heaps outside the roomblocks). Rooms were presumably living and/or ceremonial areas, which might be expected to have faunal assemblages distinct from those in outside contexts such as plazas and middens.

Indeed, for those assemblages at Tijeras with provenience information (that is, the faunas recovered during the 1970s excavations) this seems to be the case (Figure 3). Taxonomic relative abundance differs significantly between room and midden contexts (χ2 = 43.078, p = .000), with artiodactyls and turkeys more abundant in rooms, and carnivores, birds other than turkeys, and (less dramatically) lagomorphs more abundant in middens.

Figure 3 

Taxonomic relative abundance at Tijeras Pueblo by spatial context.

In contrast, spatial provenience is not available for the faunal assemblages from the 1968/1969 excavations. Taxonomic relative abundance for the fauna from these assemblages more closely resembles the 1970s “rooms” fauna (χ2 = 78.843, p = .000) than the “midden” fauna (χ2 = 143.500, p = .000), but still differs significantly from both (Figure 3). Some of these differences likely reflect the lack of screening (notably the under-representation of rodents and lagomorphs and the over-representation of artiodactyls) and, possibly, the amalgamation of faunas from both room and midden contexts. However, the predominance of carnivores in the 1968/1969 assemblage is puzzling. Without information on the provenience of these materials, it is impossible to know what is driving these patterns.

We can say, however, that middens and roomblocks were used – and fauna deposited within them – in different ways. As in assemblages derived from a hunting camp and a residential village, these inter-site differences have the potential to reveal information about past human behavior – if the information that allows analysis by context or site type is available. In a meta-analysis, if such information is not available, at best only broad-scale interpretations will be possible. At worst, as in the screening example, differences in site type may correspond with other variables in the data, leading to incorrect conclusions. For example, if half the analyzed data derive from roomblock contexts and the other half from middens and this contextual bias is not addressed, an analyst might erroneously conclude that there was a difference in subsistence patterns between sites, rather than a difference in representation between contexts. In such a case, this hypothetical meta-analysis reveals patterns that derive from the behavior of archaeologists, rather than the behavior of site inhabitants.

Documentation

Variability in screening protocols and problems in understanding site function are not limited to older data; nor are these challenges insurmountable (see for example Jones 2013b). However, lack of documentation can transform both of these otherwise manageable issues into serious problems (Atici et al. 2013). Many older collections are poorly-documented, which means analysts may not know what screening (if any) occurred, or the location within a site from which a collection originates; sometimes information about site function is limited as well. Not only do these challenges cause problems with data validity; they cause problems with the assessment of data validity.

A recent reanalysis of the fauna from Comanche Springs (LA 14904), a 17th century Spanish colonial habitation site located south of Albuquerque (Figure 1), illustrates the compounded problem. Comanche Springs was originally excavated in the 1970s by Frank Hibben, who published a brief report of his excavation – including faunal data – in 1985. The site was revisited in the late 1990s by Ann Ramenofsky; however we only address the Hibben collections here, so will not discuss the Ramenofsky re-excavation further.

The original publication of the Comanche Springs fauna indicates a number of identified specimens (NISP) of just under 2000, representing 21 total taxa (Hibben, Benjamin and Adler 1985). The reanalysis, however, resulted in a NISP of only 373 (total NSP or number of specimens: 2561) representing 22 taxa. One possible interpretation of this discrepancy is that the original report’s number represents NSP rather than NISP; a total number of specimens of 2000 would not be inconsistent with the reanalysis’s findings. The similar numbers of taxa reported – and overall similarities of the assemblages – at first glance seem to support this interpretation (Table 1). However, further investigation suggests a more complex picture. The reanalysis identified seven taxa not previously recorded, while six of the previously identified taxa were either identified at a lower resolution or not identified at all. While some of these differences likely reflect differences in analytical approach (particularly recording to lower levels of taxonomic resolution), the taxonomic differences raise the possibility that some of the collection has been lost. This hypothesis was supported when, in February 2015, multiple previously unaccessioned boxes from Hibben’s excavations at Comanche Springs were located in the Maxwell Museum warehouse. Some of these boxes contain fauna which may resolve the disparity between the original analysis and the reanalysis. We will not know for certain if this is the case, however, until the ‘new’ collections are examined; the fauna in these boxes may never have been previously analyzed.

Table 1

Fauna identified at Comanche Springs by Hibben and colleagues (1985) and in this project; (a) indicates taxon identified to lower resolution and (b) indicates new taxon.

Hibben, Benjamin and Adler 1985 This study

Old World fauna Mammals
     Domestic horse (Equus caballus) Carnivora
     Domestic pig (Sus scrofa)       Canis sp. (domestic dog/coyote)a
     Domestic goat (Capra hircus – Scimitar-horned goat)       Ursus sp. (bear)b
     Domestic sheep (Ovis aries – “scrub” sheep) Ungulata
     Domestic dog (Canis familiaris – a large Spanish-type of dog)       Antilocapra americana (pronghorn)
     Domestic cow (Bos taurus – Spanish longhorn)       Bos/Bison
     Oxen (large animals for draft)       Bison bison (bison)b
Wild animals       Bos taurus (domestic cow)a
     Mule deer (Odocoileus hemionus)       Capra hircus (domestic goat)
     Antelope (Antilocapra americana)       Equus caballus (horse)
     Coyote (Canis latrans)       Ovis aries (domestic sheep)
     Black-tailed jackrabbit, Desert cottontail (Leporidae)       Cervus elaphus (elk)b
     Yellow-bellied marmot, Black-tailed prairie dog (Sciuridae)       Odocoileus sp. (deer)a
     Bott’s gopher, Plains pocket gopher (Geomyidae)       Sus scrofa domesticus (domestic pig)
     Sucker (Osteichthyes) Rodentia
     Great Plains toad (Amphibia)       Geomydae (pocket gophers)
     Ornate box turtle (Reptilia)       Perognathus flavescens (apache pocket mouse)b
Birds Lagomorpha
      Domestic chicken (Gallus gallus – small Castilian race of fowl)       Sylvilagus sp. (cottontail)a
      Ducks, geese and swans – Mallard duck (Anatidae) Birds
      Red-tailed hawk (Buteo) Galliformes
      Prairie falcon (Falconidae)       Gallus gallus (domestic chicken)a
      Turkey (Galliformes)       Meleagris gallopavo (turkey)a
      Common raven (Corvidae)       Phasanidae (grouse, pheasants, and partridges)b
Falconiformes
      Falco sp. (falcon)
Strigiformes
      Strigidae (owls)b
Passeriformes
      Corvus sp. (crows)b
      Corvus corax (common raven)b
Reptiles
Testudines
      Testudines (turtles)
      Terrapene ornata (Western box turtles)

In short, despite both a publication and a reanalysis, we know very little about the Hibben Comanche Springs archaeofauna. If the unaccessioned boxes had not been located, the lack of documentation would make it impossible to know whether part of the Comanche Springs collection was missing or if the original report contained a typographical error; and we still do not know if the original publication is an accurate analysis of the complete zooarchaeological assemblage. These problems lead in turn to the question of, which data to use in a meta-analysis? Use of Hibben’s original data in a study of taxonomic composition in 17th century New Mexican archaeozoological assemblages would produce a different result than would use of the reanalysis data, but as of this writing, we have no way of knowing which analysis is more accurate.

When is there a problem?

The specific challenges posed by the Comanche Springs and Tijeras Pueblo assemblages would, of course, be only one small part of a larger meta-analysis, and if other included datasets did not have similar problems then these issues would likely not surface at all. After all, one of the benefits of meta-analyses is that issues specific to particular assemblages should in most cases be outweighed by larger-scale patterns in a dataset. As long as the variability in problematic datasets is stochastic – that is, where the error can be expected to be non-directional – such variability should not impact the overall meta-analysis. A classic example of this kind of error can be found in linear measurements. A dataset comprised of (for example) measurements of the distal breadth of cottontail humeri taken by multiple analysts will almost certainly contain inter-analyst error, but one would expect such error to be random – and thus the dataset will probably be suitable for inclusion in a larger analysis.

Sometimes, however, error is not random. Variations in taxonomic relative abundance caused by differences in screening protocol (Figure 2), by differences in site type (Figure 3), or by sampling problems (Table 1) are all directional, rather than stochastic. This means that certain questions – about taxonomic representation in the last case, and changing relative abundance in the first two – cannot be answered using collections with these issues, unless said issues are somehow addressed and resolved. Attention to the impacts of data errors is thus critical to zooarchaeological meta-analyses (for additional discussion see Peacock et al. 2012; Wolverton 2013; Wolverton, Dombrosky and Lyman 2014). We will illustrate this problem of directional difference with a case study focused on the distribution of domestic fauna of Old World origin in 17th century New Mexico.

Case study: Old World domesticates in 17th century New Mexico

When Coronado withdrew from the Southwest in 1541, one of the many legacies he left among the groups he visited was domestic animals of Old World origin. While the ultimate fate of the sheep and cattle introduced by the Coronado entrada is unknown, both documents and the archaeological record of the first permanent Spanish settlements in New Mexico (A.D. 1598) reveal a more lasting introduction. Settlers with Oñate’s 1598 colony brought with them domestic goats (Capra hircus), sheep (Ovis aries), cattle (Bos taurus), horses (Equus caballus), and pigs (Sus scrofa domesticus), all taxa new to New Mexico (Hammond and Rey 1953: 210–220). Other records suggest the colonists introduced chicken (Gallus gallus) and donkeys (Equus africanus) as well (Dunmire 2013; Pavao-Zuckerman and Reitz 2006). These introduced fauna eventually transformed both the Southwestern landscape and the indigenous peoples who lived on it; their distribution of Old World domesticates in native communities and the ways in which they were used are thus a subject of much discussion (Jones in press; Ramenofsky 1996; Ramenofsky, Wilbur and Stone 2003; Spielmann et al. 2009). However, domestic taxa also provide information about Spanish identity in early colonial New Mexico, a topic not yet well understood.

Compared to other New World Spanish settlements, New Mexico was characterized by physical and political isolation from the central Spanish government (Brooks 2002). The main route north, the Camino Real, consisted of 2,560 kilometers of land that traversed the Sierra Madres, Chihuahua Desert, and the Rio Grande Valley (Preston, Preston and Esquibel 1998). The perilous nature of this journey meant that trade caravans – the source of both European goods and outside correspondence – arrived in New Mexico only irregularly (Weber 1992). The lack of access to goods and marginal location of the colony made New Mexico overwhelmingly and collectively poor in relation to other colonial territories; historians argue, based on this collective poverty, that there was little distinction between classes in early colonial Spanish New Mexico (Kessell 2008).

A diet containing fauna of Spanish origin was, by most accounts, a critical component of colonial Spanish identity in the New World. Pork and mutton were the most common meats in Early Modern Iberian Spain, and correspondingly wheat, pigs, and sheep were considered central to a civilized Spanish life (Trigg 2005). However, given New Mexico’s isolation from the rest of the Spanish New World, maintaining a “civilized Spanish life” far from supply trains may have been a challenge. The distribution of domestic taxa across Spanish sites in 17th century New Mexico should thus reflect the degree of isolation of the inhabitants.

Methods and materials

We examined published and grey-literature faunal data from 14 Spanish-affiliated 17th century archaeological sites from New Mexico (Figure 4). This use of previous research results allowed us to undertake a regional scale analysis; however, the potential sources of bias in these data are many (Tables 2 and 3). The assemblages considered here differ in screening and collection procedures, excavated context, scale of excavation, and existing documentation (Table 2); and they often vary not only between sites, but within assemblages. In addition, taxonomic data from these assemblages were recorded at different levels: for some (e.g., the Sanchez Site, LA 20000) only presence/absence data were published, while others (i.e., Santa Fe, LA 5400) were presented in a full analysis including identification protocol, NISP, and Minimum Numbers of Individuals (MNI). Some contained extremely small samples (i.e., Agua Fria Schoolhouse, LA 2) while others had NISPs of over 1000.

Figure 4 

Distribution of domestic taxa in 17th century Spanish archaeological sites.

Table 2

Potential sources of analytical bias in assemblages included in the 17th century New Mexico Old World domesticates project; italicized entries indicate the subset included in quantitative analysis.

Site name Screen size Context Documentation Sources

Yunque (LA 59) None – 1/4" Roomblock, plaza, possible church Moderate Gabe and Jones 2014
Comanche Springs (LA 14904) None – 1/8" Roomblock, exterior features Multiple excavations, various Gabe and Jones 2014
Santa Fe (LA 5400) 1/4" ? Exterior features, midden Multiple excavations, various Snow & Bowen 1995
Metzler Ruin (LA 103997) None – 1/4" Roomblock, exterior features Low Gabe and Jones 2014
Cochiti Springs (LA 34) 1/4" ? Roomblock Good Snow 1971
Las Majadas (LA 591) 1/4" ? Roomblock, exterior features, midden Good Harris 1973
LA 16768 ? Roomblock, exterior tests Low Levine et al. 1985
Sanchez Site (LA 20000) None – 1/4" ? Roomblock, exterior features, midden Multiple excavations, various Trigg 1999
Agua Fria Schoolhouse (LA 2) ? Roomblock Multiple excavations, various Payne 1999
Signal Site (LA 9142) 1/4" ? Roomblock, exterior tests, exterior features Good Alexander 1971
LA 13659 1/4" ? Cave? ? Allen 2004
LA 65005 1/4" Roomblock, exterior tests Excellent Moore 1993
Casa Quemada (LA 4955) 1/4" Roomblock, midden Moderate Trigg 1999
LA 12117 1/4" ? Cave? ? Allen 2004

Table 3

Distribution of Old World domesticates across 17th century New Mexican sites; italicized entries indicate the subset included in quantitative analysis.

Site name Type Pig Chicken Equid Cow Sheep/Goat total NISP total MNI Recent analysis?

Yunque (LA 59) Capitol x x x x x 286 yes
Comanche Springs (LA 14904) Large x x x x x 373 yes
Santa Fe (LA 5400) Capitol x x x x x 1001 53 yes
Metzler Ruin (LA 103997) Large x x x x x 265 yes
Cochiti Springs (LA 34) Large x x x 20 no
Las Majadas (LA 591) Large x x x 192 63 no
LA 16768 Small x x 107 no
Sanchez Site (LA 20000) Large x x x x x no
Agua Fria Schoolhouse (LA 2) Small x x 20 no
Signal Site (LA 9142) Small x x no
LA 13659 Small x x 18 no
LA 65005 Small x x 7 no
Casa Quemada (LA 4955) Large x no
LA 12117 Small x 9 no

We considered all the assemblages, but we took several steps to reduce the impact of these biases. We excluded sites with a NISP of under 100 or a MNI of under 50 from quantitative analyses (though we did consider them qualitatively) because such low samples would almost certainly distort results. Accounting for screening and contextual variability was more challenging, as we did not always have access to information about the context from which our samples were drawn (i.e., documentation was lacking). However, we discovered that by excluding those assemblages with a NISP of under 100 or a MNI of under 50 from quantitative analyses, we also excluded those sites with the least documentation about context and screening protocol; those that remained in the set are roughly comparable (Table 2).

We focused our analysis on large domestic taxa – the specimens most likely to be recovered and identified correctly, even with variations in screening. For each assemblage we tallied the presence of five introduced taxa of Old World origin: pig, chicken, equids, domestic cow, and sheep/goat. While the identification of domestic cattle can pose problems in older datasets (see Jones in review), in this study we accepted such identifications at face value. Taxonomic richness is strongly influenced by sample size (Lyman 2008) and this of course poses a problem for this analysis. We tested for a correlation between sample size (total NISP) and the number of domestic taxa for those collections for which NISP was available, and found no significant relationship (r = 0.537, p = 0.272). With a maximum number of domestic taxa of five and a relatively small sample, however, we cannot rule out the possible influence of sample size on these analyses.

We used ArcGIS to measure several proxies for isolation: distance to the Camino Real, distance to the capitol, and distance to the nearest known (roughly) contemporary Spanish household. We also used a classification we call “site type” as another proxy for isolation. Making an assumption that the number of households within a site will roughly correspond with isolation (for small households) or lack thereof (for multiple households) within the larger Spanish administrative core, we divided the 14 sites into 3 site types: 2 capitols (government centers with multiple households), 6 large (extended family households comprised of structures with more than 5 rooms), and 6 small (probable single family households with 4 or fewer rooms).

For assemblages with a sufficient sample size, we tested for associations between the number of domestic taxa present and these proxies for isolation. We used Spearman’s rank-order correlation (VanPool and Leonard 2011), a nonparametric test measuring the strength of association between two ranked variables.

Finally, we qualitatively considered the role of age of analysis on the number of taxa identified. Zooarchaeological assemblages identified after 1990 we classed as a “recent analysis”; those identified prior to 1990 we classified as an “older analysis.”

Results

Domestic taxa in this dataset do seem to be distributed non-randomly (Table 3). The most common domesticate represented is sheep/goat (present at 12 sites), followed by cow (10 sites). The least widely represented is pig (5 sites). These results correspond with what is known from the documentary record, which suggests widespread sheepherding by Spanish New Mexicans, but relatively little investment in pig farming (Barrett 2012; Dunmire 2013).

Within the subset of sites used for quantitative analysis, there also seems to be an association between site type and number of domestic taxa, with larger numbers of domestic taxa at the capitols, an intermediate number at “large” sites, and fewer at “small” sites (rs = 0.775 p = 0.041). The other proxies for isolation, however, showed no significant correlation with number of domestic taxa (distance to Camino Real: rs = 0.228 p = 0.623; distance to capitol: rs=0.335 p=0.462; distance to nearest contemporary Spanish habitation: rs = 0.060 p = 0.971). The lack of correspondence between our proxies is both pronounced and surprising. The fact that site type, the least direct of our proxies, is the only one with a significant association suggests the relationship between site type and domestic taxa probably does not solely reflect isolation from the wider Spanish community, but that some other factor is at play – perhaps site function, but also perhaps an analytical bias. The level of comparability between the contexts from which these assemblages are drawn is difficult to address with the available documentation (Table 2), and this may be influencing these results.

The “age of analysis” figures hint at an additional possible analytical bias in these data (Table 3). 80 percent (or four out of five) of the assemblages containing all five possible domestic taxa are “recent analyses,” while all seven of the assemblages with two or fewer domestic taxa were identified prior to 1990. These numbers are too small to test for significance, but they are nonetheless suggestive.

Discussion

While the non-random distribution of domestic taxa is unsurprising, the lack of correspondence between our proxies for isolation is intriguing. Government supply shipments beginning in the mid to late-17th century typically included livestock to supplement mission supplies in the colony (Dunmire 2013), and these supply trains ran along the Camino Real to the missions and the capitols. If Spanish residents in dispersed settlements were less “Spanish” in their diet, as has been suggested (Pavao-Zuckerman and Loren 2012; Weber 1992), we would expect to see an association between the three distance measures, as well as site type, and the number of domestic taxa.

This is not the pattern present in these data. Instead there is a significant correlation between site type (the weakest of our proxies) and number of domestic taxa, and no association at all between domestic taxa and the distance measures. Site type thus seems to be measuring something other than simply isolation from the Spanish community (though, as distance and isolation are not equivalent, we cannot dismiss isolation with these data). There may be some difference in site function between our site types, and there may be a contextual bias in these data as well. We are exploring both these possibilities further in future research. These data also suggest the need for more reanalyses. While sample size constraints prevent a thorough test of the impact of newer analyses on these data, the apparent relationship between age of analysis and number of domestic taxa is striking.

On a more positive note, our analysis suggests that in many cases, biases may be linked. In our study, the assemblages with small sample sizes were often also those with little documentation of screening protocol and excavated context. Though our only solution was to exclude these assemblages from quantitative analysis, we did find that dealing with one of these sources of bias meant dealing with several.

Conclusions

Both the New Mexican Old World domestic fauna case study and the examples from the Tijeras Pueblo and Comanche Springs archaeofaunas point to benefits and challenges posed by incorporating older collections into zooarchaeological meta-analyses. On the one hand, such collections constitute a rich source of information. Particularly in regions like the American Southwest, ignoring older zooarchaeological collections would result in a severely limited sample size. However, such collections contain biases, sometimes severe ones, as illustrated here, and working with them means grappling with the ways in which such biases constrain one’s analytical options.

Transparency is thus critical when making use of such collections. Whenever one incorporates older material or information, biases should be taken into consideration in the analysis (which may mean avoiding some powerful – but inappropriate – statistical analyses) and discussed fully in publication. While the issues we discuss in this paper are far from exhaustive, they are common to many (if not all) older collections. Changes in archaeological practice – here represented by changes in screening protocols – can alter the composition of archaeofaunas, and if such changes correlate with differences in research interests (for instance, if archaeologists from different eras preferentially focused on sites from different time periods, as is common) then spurious patterns can appear in meta-analyses. It is thus important to account for such changes in archaeological practice before analysis.

Second, assessing the factors contributing to the composition of an older collection relies on research with original notes and reports. In the Comanche Springs case, the lack of documentation means we are currently unable to resolve the differences between the original report (Hibben, Benjamin and Adler 1985) and the reanalysis (although identification of newly discovered material may settle this issue in the future). A similar lack of information about the 1968/1969 Tijeras Pueblo excavations makes understanding the faunal composition from those collections impossible. Documentation is a critical component of understanding the biases older collections bring to a meta-analysis.

Finally – and perhaps most importantly – all collections have their biases, and these must be taken into account in larger analyses. Many older collections, due to biases stemming from differences in screening protocol or lack of contextual information, will not be amenable to studies involving comparative taxonomic relative abundances, but presence/absence analyses might be productive. For other collections, even presence/absence analyses may be problematic. In short, one must consider the limitations of a dataset – what questions can and cannot be answered – before proceeding with analysis.

Much of this paper has been dedicated to exploring the issues that arise in the use of older collections in meta-analyses, but we also believe strongly in the importance of using older collections and older data in research. In many regions such collections are plentiful and a rich source of information, if their limitations are addressed in research. The issues with older collections should not be taken as a reason to exclude them from analysis, but rather as an opportunity to engage with data validity more broadly (e.g., Wolverton 2013). In addition – results of the case study presented here notwithstanding – it would be unwise to consider more recent collections necessarily superior by virtue of their age. As we discussed at the beginning of this paper, some newer collections are as plagued by data validity issues as older ones. Strong meta-analyses will address issues of data validity regardless of the age of the collections they contain.

Competing Interests

The authors declare that they have no competing interests.