tDAR digital antiquity


Heartbleed Response and tDAR Security

Last week, internet security experts announced a major flaw ‘heartbleed‘ in commonly used encryption software (OpenSSL).  We take the security and safety of data entrusted to tDAR seriously.  We wanted to take a moment and both outline what we’ve done regarding the ‘heartbleed’ bug, but also take a moment to discuss how we protect your data. 

Was tDAR affected?

Like much of the internet, tDAR’s infrastructure was running a version of OpenSSL that was affected. We have seen no evidence that this bug was exploited.  The Digital Antiquity staff took immediate action on a number of fronts including:

  • immediately patching each of the affected servers within hours of the announcement
  • working with our vendors to re-issue the SSL certificates that may have been compromised in the process

How do we handle server security?

The security of client’s data is of critical importance to us.  We take a number of standard approaches to managing the security of tDAR.  These include:

  • Limiting access to each of our machines and running and testing firewalls that limit this access
  • Running Enterprise focused OS versions which tend to be more conservative from a security standpoint and undergo more testing.
  • Patching our servers regularly, usually daily.
  • Limiting the services and applications running on our machines.
  • Coordinating with external IT specialists in the University and elsewhere to test our servers for common vulnerabilities.

How do we handle application security?

  Beyond testing and patching our servers, we also test the application regularly.

  • We work with external IT specialists to run common security analysis tools on our software to identify vulnerabilities.
  • We try to hack our own software.
  • We run over 1000 tests on our software prior to release, many of these are focused around rights and permissions. A number of these tests also attempt to perform actions that a user would not have rights to perform, eg. escalate permissions.

Preserving Archaeological Legacies: Turning a Citation into a Resource

In 2011 the Center for Digital Antiquity used information about archaeological reports found in the National Archaeological Database (NADB) to creates over 350,000 tDAR citation records. These new tDAR records improved this information with enhanced metadata and a display of geographic information that enable for easier discovery and access. In tDAR these records can be edited and improved; for example, if a digital file of the report described in the citation record is available, it can be uploaded and added to tDAR, thereby greatly enhancing accessibility to the information.

Recently David Hughes discovered the tDAR record for a report he co-authored in 1987: The Courson Archeological Projects, 1985 and 1986: Final 1985  and Preliminary 1986. The report documents the results of fieldwork done at the sites Courson A (41OC26) and Courson B (41OC27) as well as the almost pristine Kit Courson site (41OC43), and it also covers the history of archaeological work done at an area known as the ‘Buried City’. Anyone interested in the history of  archaeological practice during the early twentieth century–and who isn’t?– will find this section very engaging, as this introduction indicates (Hughes & Hughes-Jones 1987, pp. 7):

Many interesting human details about archeological investigations are rarely published. The stories exist in field notes, correspondence, anecdotes and rumors about the personal and professional relationships of those involved, the behavior of the crew, the weather, the attitudes of the local landowners, and vehicle breakdowns and other nuisances of field work. Particularly for the Moorehead expeditions, there is more to the history of archeological investigations at the Buried City than appears in published reports. Part of the story lies in the methods of archeology some 60-80 years ago, and part lies in the relationship of two strong-willed scholars of different backgrounds and, apparently, different values. The untold story explains a significant loss of data that occurred even before the passage of time between Moorehead’s last expedition in 1920 and the current project in 1985. This story is so important to the history of archeology on the Courson Ranch that we present it in some detail here. 

Hughes contacted Digital Antiquity and offered to scan a copy he had of the report, which he then sent to us. We were able add the digital copy to the existing tDAR record and add additional metadata. This means that this once hard to access record of archaeological practice  is now easily find-able and accessible thanks to NADB, tDAR, and Hughes.

We’d like to encourage other archaeologists and tDAR users to please get in touch if they have access to a copy of one of the  citation-only records already in tDAR.  A digital curator can work with you to add the file to the repository at no cost.  Do you or your organization have multiple reports or a legacy of archaeological work that you want to see preserved? Please get in touch to learn about the services that Digital Antiquity can provide so you can turn your archaeological materials into a long lasting legacy.


Don’t Delay! The Importance of Good Digital Curation Now!

FPMcManamon


Archaeologists are up-to-their-ears in digital data and, just like physical artifact collections and paper records, these digital data must be curated properly so that the information they contain is not lost.  But, what does this mean?  What is good digital curation?  Well, it is more than storing digital data in iCloud or a Dropbox account, neither of which provide for long-term preservation, data-sharing, or future use of the data.  And, it isn’t simply putting your data on a website and hoping that colleagues who might be interested will find it and use it.


The level of understanding of what comprises digital curation and why it is important within the contemporary archaeology community is reminiscent of the situation a generation ago regarding the curation of physical collections and records from archaeological investigations.  Then, many archaeologists did not consider how the physical collections of artifacts, samples, and records they created in each field investigation would be curated.  These concerns were left to be dealt with by museum curators or not at all.  Now, planning for archaeological investigations must take account of how and where physical collections and records will be curated.  Archaeologists are required to consider this aspect of their archaeological projects.  Similarly, planning and appropriate treatment of digital data as a normal part of archaeological investigations is essential to ensure that these results of studies are discoverable, accessible, and preserved for future use. An important challenge for the archaeological community and individual archaeologists is how to bring digital curation into archaeological practice without waiting for another generation to pass.  We need to shorten the period within which proper digital curation and preservation of archaeological data becomes a regular part every archaeological project.


The Digital Curation Centre, a national authority on the subject in the United Kingdom, describes digital curation as “maintaining, preserving, and adding value to digital research data.”  To flesh out these terms a bit, one can describe good digital curation as:

  • organizing a project’s digital files logically for efficient administration, management, and research;
  • creating detailed and “rich” metadata describing the file contents and linking this metadata directly with the files;
  • uploading files to a repository (we would recommend tDAR for archaeological data) where they can be discovered and appropriately accessed; and,
  • managing files in the repository to ensure their long-term availability for future uses. 


Detailed guidance about digital curation is available, for example, the Center for Digital Antiquity and the Archaeology Data Service provide quite body of methodological, practical, and technical information about organizing and treating digital data on their webpages, Guide to Good Practice.  Last year these organizations published a handbook with basic guidance about good digital curation methods and techniques, Caring for Digital Data in Archaeology, available from Oxbow Books.


Now, word is spreading wider.  The recently published Encyclopedia of Global Archaeology, by Springer, includes an article: “Digital Archaeological Data: Ensuring Access, Use, and Preservation.”  A preprint version of this article is available in tDAR, where it can be viewed and/or downloaded by registered tDAR users.


There are positive developments based on broader national and international efforts.  For example, the US government recent policies requiring improved access to research data and information generated by government agencies.  Another positive development is the greater emphasis on requiring good digital data management by granting agencies like NSF and NEH.  Academic and scientific publishers, including the Society for American Archaeology and Elsevier, are emphasizing making data used in published articles available in digital formats. All of these general developments are moving in the right direction for improving the inclusion of good digital curation as part of contemporary archaeological practice.  With all this positive background, there is no reason for individual archaeologists or agencies responsible for archaeological information to delay the incorporation of good digital curation into their own work.  Let’s not wait for 20, or 25, or 30 years for digital curation to become part of archaeological good practice.  We will have lost much too much data and information if we delay.


tDAR Software Update (knap)

Digital Antiquity is proud to announce the release of “Knap,” the latest release of tDAR.  The “Knap” release required the tDAR staff to take a step back and review the entire application from a number of major perspectives including, performance, security, data storage, and user-experience.  Much of this work helps to establish features that will be available in future releases for you to enjoy.

 We focused on a number of major areas of the code including:

  • Improved application security
  • Clearer error messages, and better in-form validation
  • Increased performance of the entire web-application (faster searches and page loading)
  • Better display on mobile  devices
  • Ability to add ORCID Identifiers to your user account
  • Improved results for auto-completes with many results, especially when searching for people
  • Improved validation and error messages for bulk uploads.
  • Bulk uploads now support data sets
  • Resources can now inherit individual and institutional roles from projects
  • File Descriptions are now printed on cover-pages, which may be useful for redaction notes
  • Display of new and popular items on the explore page
  • The user-registration page was simplified
  • Pagination options were added for the column metadata screen
  • Table and column relationships are display for MS Access Databases
  • Fixed parsing issues with converted OWL ontologies, now maintaining import order, and improving duplicate checking

Today is World Backup Day…Are Your Valuable Archaeological Data Backed Up?

There are lots of reasons to backup your data, including protection from loss, accidental damage, or device failure, or to simply have access to older versions in case of mistakes.  Good backup practices require maintaining multiple copies of the data, ideally in physically different locations.  If you’d like more information on backup procedures (or horror stories!), review the Guides to Good Practice

Importantly, storage media (CDs, Flash Drives, External Hard Drives, etc.) are great short-term backup solutions, but are not designed to protect your information in perpetuity.  Burned CDs have a lifespan of only a few years[1], and hard and flash drives have a limited number of write-cycles[2]

Are you looking for a more long-term solution? Celebrate World Backup Day by archiving your archaeological information in tDAR!  tDAR is so much more than simple file storage–the repository offers a full archival solution.  

Digital files in tDAR are 

  • protected from catastrophic loss; 
  • accessible from anywhere with an internet connection;
  • always available in up-to-date file formats so you can open and use your files today and long into the future;  
  • associated with rich, archaeologically specific metadata for easy search and discovery.

Don’t wait!  Upload your digital files today before it is too late!

 

 


Three Ways to Connect with Digital Antiquity Staff at the SAA Meetings in Austin

Going to the SAA Meetings in Austin?  Connect with Digital Antiquity staff to learn more about Digital Preservation, tDAR, and Digital Antiquity by:

 
Attending our forum on Digital Preservation and Curation for Archaeological Data – Thursday, 6PM

  • You’ll hear from public agency archaeologists, CRM firms, researchers, teachers, and archivists who discuss the successes and challenges of digital preservation and data reuse.


Scheduling an appointment with one of our digital curation experts at SAA.  Click here to schedule your appointment! We can help with specific problems or questions, such as:

  • New to tDAR?  Ask for a quick guided to tour!
  • Are you an SAA student member and need help deciding what to archive with your tDAR credit? 
  • Are you planning a new archaeological project and want to ensure good digital data archiving from the outset?
  • Do you want to learn how to budget properly for digital curation using tDAR in responses to RFPs?
  • Are you interested in learning more about data integration for synthetic research? 
  • Are you afraid your digital archaeological legacy is at risk, but don’t know where to start? 
  • Would you like to see how easy it is to add a file to tDAR? 
  • Do you know how much to budget for digital data management for your next grant or project proposal?
  • Do you need help organizing managing your personal, project, office, or agency archaeological files?

 
Visiting us at the Digital Antiquity booth in the exhibit hall anytime.

  • Our booth number is 511 and we will be open from 9 AM – 5 PM.
  • You can say hello, and enter to win one of our daily giveaways!


 Hope to see you there!


Shaw AFB and Avon Park AFR Archaeology Archives now in tDAR

In partnership with the United States Air Force (USAF), the Shaw Air Force Base (Shaw AFB) in South Carolina and Avon Park Air Force Range (Avon Park AFR) in Florida archaeology archives were recently added to tDAR.  Each archive contains documents, images, and other data from archaeological and other cultural resource research conducted at both bases.  The creation of these digital archives is part of a pilot program to investigate the feasibility of the USAF using tDAR as a long-term repository for archaeological information important for the management and protection of important archaeological resources on USAF bases.  The records in the Shaw Air Force Base Archaeology Archive are organized as a collection within tDAR which includes 512 files.  The Avon Park Air Force Range Archaeology Archive also is organized as a tDAR collection and includes 219 files.

Most of the information in the archives is generally available.  However, due to confidential information, mainly specific site locations, included in some of the files, the collections’ material are accessible according to three  categories depending on their content.  Confidential records contain sensitive USAF information and are available only to the USAF officials responsible for the archaeological resources or others authorized by these Air Force officials; confidential with redacted copy available are files from which USAF sensitive information has been removed and a redacted version is available to registered tDAR users; and, available to all users are files  that contain no confidential information and are available to all registered tDAR users.

The USAF digital archives project demonstrates how staff at the Center for Digital Antiquity can work under contract or cooperative agreement with public agencies to provide digital curation services directly to agencies.  Some of these services include: organization of materials, drafting of metadata, examining files for potentially confidential information, and uploading files to tDAR. The USAF project to date has been funded by a contract administered through the CRM consulting firm GMI (now part of Versar).  USAF staff worked closely with experts at Digital Antiquity to review draft metadata and redacted versions of files before final versions were made public in tDAR.  At Digital Antiquity we look forward to working with the USAF on more digital archives for facilities and with other agencies on similar projects.

Have questions about the USAF pilot, or a similar project you would like to start, contact us.


Tips: Troubleshooting Data Sets in tDAR

Over the past few years we’ve seen a lot of data sets go into tDAR. In that time, we’ve learned a great deal about what makes a data set ready for archiving, and conversely, what some of the common problem-spots are. Within tDAR, we try to catch many of these errors and provide users with warnings. We want to make the uploading process as easy as possible! We’ve put together a few tips to avoid the most common issues we see when archiving a CSV, TAB or Excel data set. Click here to view all the tips.
Have you discovered any shortcuts that make it easier for you to upload data sets? We would love to hear them! Send us an email at comments@tdar.org with your tips, comments or concerns.

tDAR Poised to Help with Grand Challenges

In a recently published paper in American Antiquity, Kintigh and colleagues describe an effort to identify “What are archaeology’s most important scientific challenges?” This question was posed to the archaeological community to crowd source key themes, and the results were used to inform and augment the topics developed by an esteemed group of scholars.  The top 25 “grand challenges” they identified are replicated at the end of this post.

What the authors argue is most needed to address these important research questions is not more data (though, undoubtedly some new field work will be undertaken), but rather, a discipline-wide effort to locate, synthesize, and interpret the extensive amounts of data that have been collected through extensive archaeological efforts to date.  tDAR’s data integration tools have been developed and refined with these tasks in mind, and the repository is capable of serving as a storage facility for the data and the supplementary information that support them. 

Data sets archived in tDAR can include detailed column metadata describing the data so that researchers unfamiliar with the data set are still able to understand and reuse the data.  Furthermore, multiple discrete data sets can be integrated into large, synthetic data sets using tDAR’s data integration tools.  In addition, data sets in tDAR are afforded the myriad other benefits to being archived in tDAR—archaeologically specific metadata and long-term preservation with forward migration to ensure that data files are accessible and usable long into the future. 

Get started today!  Add your data sets to tDAR, find others, and begin synthetic analysis!

 

Archaeology’s Grand Challenges

Emergence, communities, and complexity

  1. How do leaders emerge, maintain themselves, and transform society?
  2. Why and how do social inequalities emerge, grow, persist, and diminish, and with what consequences?
  3. Why do market systems emerge, persist, evolve and, on occasion, fail?
  4. How does the organization of human communities at varying scales emerge from and constrain the actions of their members?
  5. How and why do small-scale human communities grow into spatially and demographically larger and politically more complex entities?
  6. How can systematic investigations of prehistoric and historic urban landscapes shed new light on the social and demographic processes that drive urbanism and its consequences?
  7. What is the role of conflict—both internal factional violence and external warfare—in the evolution of complex cultural formations?

Resilience, persistence, transformation, and collapse

  1. What factors have allowed for differential persistence of societies?
  2. What are the roles of social and environmental diversity and complexity in creating resilience and how do their impacts vary by social scale?
  3. Can we characterize social collapse or decline in a way that is applicable across cultures, and are there any warning signals that collapse or severe decline is near?
  4. How does ideology structure economic, political, and ritual systems?

Movement, mobility, and migration

  1. What processes led to, and resulted from, the global dispersal of modern humans?
  2. What are the relationships among environment, population dynamics, settlement structure, and human mobility?
  3. How do humans occupy extreme environments, and what cultural and biological adaptations emerged as a result?
  4. Why does migration occur and why do migrant groups maintain identities in some circumstances and adopt new ones in others?

Cognition, behavior, and identity

  1. What are the biophysical, sociocultural, and environmental interactions out of which modern human behavior emerged?
  2. How do people form identities, and what are the aggregate long-term and large-scale effects of these processes?
  3. How do spatial and material reconfigurations of landscapes and experiential fields affect societal development?

Human–environment interactions

  1. How have human activities shaped Earth’s biological and physical systems, and when did humans become dominant drivers of these systems?
  2. What factors drive or constrain population growth in prehistory and history?
  3. What factors drive health and well-being in prehistory and history?
  4. Why do foragers engage in plant and animal management, and under what circumstances does management of a plant or animal lead to its domestication?
  5. Why do agricultural economies emerge, spread, and intensify, and what are the relationships among productive capacity, population, and innovation?
  6. How do humans respond to abrupt environmental change?
  7. How do humans perceive and react to changes in climate and the natural environment over short- and long-terms?

 

Kintigh, Keith W., Jeffrey Altschul, Mary Beaudry, Robert Drennan, Ann Kinzig, Timothy Kohler, W. Frederick Limp, Herbert Maschner, William Michener, Timothy Pauketat, Peter Peregrine, Jeremy Sabloff, Tony Wilkinson, Henry Wright and Melinda Zeder 

(2014)  Grand Challenges for Archaeology. American Antiquity 79(1):5-24.

Kintigh, Keith W., Jeffrey H. Altschul, Mary C. Beaudry, Robert D. Drennan, Ann P. Kinzig, Timothy A. Kohler, W. Frederick Limp, Herbert D. G. Maschner, William K. Michener, Timothy R. Pauketat, Peter Peregrine, Jeremy A. Sabloff, Tony J. Wilkinson, Henry T. Wright and Melinda A. Zeder 

(2014)  Grand Challenges for Archaeology. Proceedings of the National Academy of Sciences 122:879-880.


tDAR in the News

A January 16th blog post on AWOL – The Ancient World Online – has highlighted the University of Pennsylvania Museum of Archaeology and Anthropology collection in tDAR.  This collection includes supplementary information associated with a number of their print publications.  The geographic and cultural range covered is quite vast—from the South Pacific to Mongolia to Eastern Crete!  Datasets, images, and documents from these projects and many more are available to download in tDAR now. 

The National Park Service December 2013 Archeology E-Gram directs interested readers to tDAR to view resources related to the Antiquities Act.  tDAR has over 40 publicly available resources related to this important preservation law. 

Angela Huster used her tDAR credits earned as part of her SAA student member benefit to publish data associated with her recent article entitled Assessing Systematic Bias in Museum Collections: A Case Study of Spindle Whorls in Advances in Archaeological Practice: A Journal of the Society for American Archaeology.  If you have data in tDAR associated with a published article let us know!