We had a busy year in 2015, tDAR continued to grow with significant contributions from the North Atlantic Biocultural Organization , US Air Force, and US Army Corps of Engineers. tDAR had two major software releases, Munsell and Neolithic which completely redesigned the Data Integration interface to make it easier to use, expanded access to usage statistics, and updated the visualization and mapping interfaces for tDAR.

We are continuing our work, begun in earnest last fall with the Corps of Engineers and the US Air Force on digital archaeological archives for their bases and other facilities. We are also still working with the Phoenix Area Office of the Bureau of Reclamation and Midwest Archeological Center of the National Park Service on their rich archives of archaeological material.

As part of our continuing agreements with Archaeological Institute of America (AIA) and Society for American Archaeology (SAA), we ran workshops highlighting best practices in digital curation at the annual meetings for SAA and AIA in San Francisco and New Orleans respectively. We also continue to provide student members with a number of no cost uploads for contributing their data to tDAR as part of our agreement with SAA.

Content added to tDAR in 2015

resource_by_year

repository_size

Usage Statistics

While we do not maintain detailed statistics on users or use to protect user and contributor privacy, we can share some interesting aggregate data. Below are the most frequently viewed and downloaded resources.

Resources (most viewed)

Resources (most Downloaded)

Digital Antiquity is pleased to announce the 14th major release of tDAR named Neolithic. This release focuses on three major areas of the repository: (1) visualization, (2) maps and spatial data, and (3) modularization and infrastructure work. In addition, the development team has improved performance and reliability as well as making a series of smaller enhancements.

Visualizations:

We have redesigned many of tDAR’s graphs and visualizations to be more interactive and appealing. Of note, in the world map which can now be explored and shows break-downs of the different resource-types by country. The graphs on the “explore” page have also be enhanced. Finally, the ontology browser has been updated to better display larger ontologies.

Screen Shot 2015-10-30 at 5.37.27 PM

Mapping:

We’ve improved the underlying mapping libraries used in tDAR. These enhancements allow us to cluster mapped search results, improving the interface. We’ve also updated the maps on the resource pages and improved the creation and management of bounding boxes on the resource edit screen.
Screen Shot 2015-10-30 at 5.43.43 PM

In collaboration with the National Snow and Ice Data Center (NSIDC), Digital Antiquity has begun a partnership to include tDAR’s Arctic data into the NSIDC’s Arctic Data Explorer. The Arctic Data Explorer is a web application that searches for research data across a number of repositories. Including data from tDAR furthers Digital Antiquity’s mission of enabling discovery and use of digital archaeological data. The new functionality in Arctic Data Explorer ensures that when scientists search for Arctic data, archaeological data will be discovered alongside interdisciplinary data from NOAA, NASA, USGS and other repositories. Try a search now and let the Arctic Data Explorer team know what you think (contact Digital Antiquity at comments@tdar.org or NSIDC at support@aoncadis.org).

We’re proud to announce the latest release of tDAR (Munsell).  This release was focused on modernizing and re-designing tDAR’s Data Integration tool to make it easier to use and faster, but also has a number of major feature updates across the application.

Completely re-designed data integration tool:
For those unfamiliar with tDAR’s integration tool, it provides a means by which users who have uploaded data sets to tDAR can combine multiple data sets with different schema and coding conventions into a unified, shared data set. 

We’ve redesigned it to simplify the user-interface.

main integration screen
Adding data sets is simpler.
Users can search for data sets that are bookmarked or integrate-able (that is, having one column mapped to an ontology), as well as  by keyword.

selecting tables

It is easier to add integration columns via a drop-down menu.

selecting an integration column

Once data sets have been added, if they share a mapping to an ontology, users can add an “Integration Column.” Integration columns allow users to further filter the resulting data set by selecting terms.

Once an integration column has been created, users can then "filter" results by selecting values from the ontology. tDAR display's a checkmark where each data set has actual values.

Once an integration column has been created, users can then “filter” results by selecting values from the ontology. tDAR display’s a checkmark where each data set has actual values.


A new “count” column type that represents count data in data sets.

adding a count column

A screenshot of a count column being added to an integration.


It’s easier to add a display column.

adding a display column

A display column being added to the integration. Each data set can supply one column to be included in a display column.


We’ve added better documentation of the integration in the excel output.

Viewing results

The Integration Results preview screen which includes a summary table, a preview of the results, and a link to download the full results.

Additional Features:

  • Integrations can now be saved and restored.
  • We’ve improved support for larger integrations 10+ data sets
  • The results of integrations are now sorted by the selected integration columns.

Other Major Features of the release:

  • Embargoed files can now be restricted for different periods of time — 6 months, 1 year, two years, and five years.
  • Institutions can now have email addresses and be used as contacts.
  • The material keywords section now includes free-form material keywords as well as the existing controlled vocabulary.
  • Contributors now have access to their view and download statistics for the resources, billing accounts, and collections they own or have administrative rights to. These statistics show the number of views and downloads for resources over time.
  • Dedicated web pages have been introduced for keywords for improved browsing.
  • Updated “user profile” pages in tDAR.
Backend and Technical Features:
  • tDAR’s OAI-PMH endpoint now supports “sets” which represent each of tDAR’s user-generated collections.
  • The tDAR Import APIs now support replacing files and setting the same access restrictions on files as other materials.
  • A new Authentication API.
  • A number of backend changes to improve long-term sustainability.

We have a great deal planned for 2015, and it’ll start with a bang. We have a new software release almost ready to go, and our contributors continue to upload and add lots of materials to tDAR. The tDAR application had two major updates in 2014, knap and lithic. A bunch of new features were added including enhancing users’ profile pages, allowing users to duplicate resources for faster customization and upload, and dedicated pages for keywords.

New content was added to tDAR by our clients and contributors including a number of amazing 3D scans from around the world uploaded by CAST; materials from the USAF Shaw Air Force Base and Avon Park Air Force Range; and reports from PaleoResearch Institute. Closer to home, our ASU colleagues Michelle Hegmon, Margaret Nelson, and Katherine Spielmann and their students continue to add content related to their research on Mimbres ceramics and sites and Southwestern faunal collections to tDAR.

We are continuing our work, begun in earnest last fall with the Corps of Engineers and the US Air Force on digital archaeological archives for their bases and other facilities. We are also still working with the Phoenix Area Office of the Bureau of Reclamation and Midwest Archeological Center of the National Park Service on their rich archives of archaeological material.

We have partnered with the relatively new Center for Archaeology and Society here at ASU and begun collaborating with the Archaeological Institute of America, about which more in another post. We continue our work with our partners at the Society for American Archaeology on activities that promote good digital curation of archaeological data. We can’t wait to see what the rest of 2015 holds.

Content added to tDAR in 2014

ResourceBreakdown2014

ResourceBreakdown2014

2014SizeByYear

Usage Statistics

While we do not maintain detailed statistics on users or use to protect user and contributor privacy, we can share some interesting aggregate data. Below are the most frequently viewed and downloaded resources.

Most Viewed

Most Downloaded

The Center for Advanced Spatial Technologies (CAST), University of Arkansas advances research, education, and outreach about geoinformatics, geomatics, and related digital data techniques (e.g., GIS, geospatial modeling, high density survey, remote sensing, etc.). The center, in collaboration with NSF, announced the Spatial Archaeometry Research Collaboration (SPARC) program to further the promotion of geospatial research in archaeology.

CAST recently published 3D scans of several famous archaeological sites through tDAR to publish the center’s work. As part of a Digital Antiquity grant project, CAST researchers curated scan data from high-density surveys at Machu Picchu, Peru; Tiwanaku, Boliva; Ostia, Italy, and Stabiae, Italy.

CAST’s 3D scans allow you to experience the built environment of these incredible places. Moreover, the center’s data allow researchers and interested users to conduct metric analyses of scanned structures and features.

Explore Macchu Pichu, its temples, and neighboring Huayna Picchu at https://core.tdar.org/collection/27264.

Visit Tiwanaku and study the site’s monuments at https://core.tdar.org/collection/27266.

Take a digital trip to Ostia, Italy and examine its architecture at https://core.tdar.org/collection/27267.

Finally, experience Stabia, Italy and the site’s Villa Arianna at https://core.tdar.org/collection/27268.

CAST researchers created documentation to help you download and view the center’s 3D scan data sets. User documentation is available at the following web address: https://docs.tdar.org/display/TDAR/Working+With+3D+Sensory+Data+Objects. The guide also teaches you how to get started with simple metric analyses of these data sets.

We’re proud to announce tDAR’s twelfth production release: Lithic.  This release contains numerous bug fixes, performance enhancements, and security related improvements.  Some specific highlights include:  

  • Simplified sign-up, purchase, and download experiences
  • A new dashboard where non-contributing users can search and use tDAR more easily
  • Various performance and security improvements 
  • New “Contact” functionality allows users to request access to a confidential file, suggest a correction, or connect with the record owner for any tDAR record
  • Redesigned  Collection edit page to make it simpler to use
  • Improved loading speed for image galleries with lots of images
  • Improved searching:
    • Better relevancy ranking and results for resources in tDAR for a number of cases including Site Codes, pluralization, and multi-word terms
    • Collections that may be related to your search are now displayed along with search results
    • When viewing search results on a map, hovering your mouse over a result will reveal that resource’s geographic area (for  public resources only)
    • Better relevancy ranking when displaying map results
  • User Notifications on the dashboard are more personalized and dismissible
  • The ability to copy or duplicate existing resources in tDAR 
  • Improvements in data integration:
    • Simplified filter page for data integration that does not permit filtering of values that do not exist in the selected data
    • Users can now auto-select specific columns when integrating resultsUsers can now associate external DOIs with any resource type except projects

Last week, internet security experts announced a major flaw ‘heartbleed‘ in commonly used encryption software (OpenSSL).  We take the security and safety of data entrusted to tDAR seriously.  We wanted to take a moment and both outline what we’ve done regarding the ‘heartbleed’ bug, but also take a moment to discuss how we protect your data. 

Was tDAR affected?

Like much of the internet, tDAR’s infrastructure was running a version of OpenSSL that was affected. We have seen no evidence that this bug was exploited.  The Digital Antiquity staff took immediate action on a number of fronts including:

  • immediately patching each of the affected servers within hours of the announcement
  • working with our vendors to re-issue the SSL certificates that may have been compromised in the process

How do we handle server security?

The security of client’s data is of critical importance to us.  We take a number of standard approaches to managing the security of tDAR.  These include:

  • Limiting access to each of our machines and running and testing firewalls that limit this access
  • Running Enterprise focused OS versions which tend to be more conservative from a security standpoint and undergo more testing.
  • Patching our servers regularly, usually daily.
  • Limiting the services and applications running on our machines.
  • Coordinating with external IT specialists in the University and elsewhere to test our servers for common vulnerabilities.

How do we handle application security?

  Beyond testing and patching our servers, we also test the application regularly.

  • We work with external IT specialists to run common security analysis tools on our software to identify vulnerabilities.
  • We try to hack our own software.
  • We run over 1000 tests on our software prior to release, many of these are focused around rights and permissions. A number of these tests also attempt to perform actions that a user would not have rights to perform, eg. escalate permissions.

Digital Antiquity is proud to announce the release of “Knap,” the latest release of tDAR.  The “Knap” release required the tDAR staff to take a step back and review the entire application from a number of major perspectives including, performance, security, data storage, and user-experience.  Much of this work helps to establish features that will be available in future releases for you to enjoy.

 We focused on a number of major areas of the code including:

  • Improved application security
  • Clearer error messages, and better in-form validation
  • Increased performance of the entire web-application (faster searches and page loading)
  • Better display on mobile  devices
  • Ability to add ORCID Identifiers to your user account
  • Improved results for auto-completes with many results, especially when searching for people
  • Improved validation and error messages for bulk uploads.
  • Bulk uploads now support data sets
  • Resources can now inherit individual and institutional roles from projects
  • File Descriptions are now printed on cover-pages, which may be useful for redaction notes
  • Display of new and popular items on the explore page
  • The user-registration page was simplified
  • Pagination options were added for the column metadata screen
  • Table and column relationships are display for MS Access Databases
  • Fixed parsing issues with converted OWL ontologies, now maintaining import order, and improving duplicate checking

2014 looks like it’s going to be a great year, we’re already hard at work preparing tDAR for new software features, and working with clients to upload documents, data sets and images into tDAR.  tDAR grew quite a bit in 2013, we had two major software updates (in situ, and jar) including: a new face for tDAR, we added the ability to upload geospatial data into tDAR, and added new features such as enhancing the creator pages in tDAR to include related keywords and collaborators among many others.

Almost 10,000 new items were added to tDAR in 2013 including GIS data from Tikal, a large collection of images of Mimbres Ceramics, and reports from the Permian Basin, the Anasazi Origins Project, Colonial MobileDyess Air Force Base, and CRM Reports from Brockington & AssociatesHartgen Archaeological Associates, and PaleoResearch Institute

resource_breakdown_year

resource_type_by_year_2013

The repository now archives almost 600 GB  of content, nearly triple the size of the archive in 2012.
size_by_year

Usage Statistics

While we do not maintain detailed statistics on users or use to protect user and contributor privacy, we can share some interesting aggregate data.   Below are the most frequently viewed and downloaded resources.

Most Viewed

Most Downloaded