Introduction to the platform: Difference between revisions

mNo edit summary
No edit summary
(11 intermediate revisions by 3 users not shown)
Line 1: Line 1:


== What is the Folger Digital Asset Platform? ==
== What is Miranda? ==


The Folger Digital Asset Platform (the D.A.P. or DAP) is an open source software system (licensed under the GNU General Public License v3.0)  that allows for the storage of arbitrary binary files and assets and their associated metadata and inter-asset relationships. It has been designed to allow for flexible definition of the kinds of assets it stores, flexible binary file asset storage, enterprise class search, and is built from the ground up to leverage APIs.  The DAP has an import component for rapidly ingesting data, an indexing service to manage what metadata is made available for public search, a GraphQL API for asset consumption, and a microservice to publicize configured asset types and their validation schemas.   
Miranda is a digital asset ''platform,'' a foundation for the Folger's digital collections and one upon which further applications, publications, and initiatives can be built. Miranda's goal is to improve discovery and access in libraries, from making it easier for all users to explore what the Folger has to offer, to improving tools for staff to provide context and information, to increasing efficiency and cost-effectiveness in our digital storage and hosting.   


Miranda is being built to help fill a need in the mid-sized cultural institution space that is currently not well served by existing open or closed source solutions.  As "born digital" or "digital native" types of content such as databases, blog posts, mobile applications and the like become more commonplace, the existing systems that libraries and cultural institutions have relied on are either too limited, too expensive, or too inflexible to fit our institution's particular needs. 


=== Why is it being built? ===
Miranda is an open source software system (licensed under the GNU General Public License v3.0)  that allows for the storage of arbitrary binary files and assets and their associated metadata and inter-asset relationships.  It has been designed to allow for flexible definition of the kinds of assets it stores, flexible binary file asset storage, enterprise class search, and is built from the ground up to leverage APIs.  The DAP has an import component for rapidly ingesting data, an indexing service to manage what metadata is made available for public search, a GraphQL API for asset consumption, and a microservice to publicize configured asset types and their validation schemas. 


This project is made possible through the generous support of the [https://www.mellon.org Andrew W. Mellon Foundation]. Miranda's platform has been designed and built by the [https://parsonstko.com/ ParsonsTKO] firm in partnership with the Folger; the WordPress multisite installation has been built by [https://sitesbycoop.com/ Sites by Coop]. 


The DAP is being built to help fill a need in the mid-sized cultural institution space that is currently not well served by existing open or closed source solutions.  In particular as “born digital”/”Digital native” types of content such as databases, blog posts, mobile applications and the like become more commonplace the existing “record” management systems that libraries and cultural institutions have relied on are either too limited, too expensive, or too opinionated to fit mid-market organizations needs.  
For an overview of the platform, the development process, and the project's ethos for cultural insitutions who may be interested in replicating or modeling the platform for their collections, see the [https://wiki.folger.edu/collections-architecture-portal/File:Miranda_platform_white_paper.pdf Miranda white paper].  


==== How Miranda helps the Folger: ====


==== Key business areas this impacted for the Folger included: ====
* '''Enhanced remote access''' -- Not everyone can physically travel to the Folger in Washington, DC.  Existing tools were designed to facilitate discovery of assets but not as helpful in the consumption of these assets.  
 
* '''Audience expansion''' -- The Folger currently serves an audience of over 2 million people annually, online and in person. However, many audiences could not be easily targeted with library- or institution-specific data formats, data exchange standards, and other types of technical roadblocks of high learning curves.  By developing a system utilizing widely adopted, well-documented standards and solutions, the DAP makes it much more likely that new audiences and organizations can and will leverage Folger assets.
* '''Enhanced remote access''' -- Not everyone can physically travel to the Folger Shakespeare Library in Washington, DC.  Existing tools were designed to facilitate discovery of assets but not as helpful in the consumption of these assets.  
* '''Audience expansion''' -- The Folger currently services an audience of around 1 million people annually. However many audiences could not be easily targeted with library or institution specific data formats, data exchange standards, and other types of technical roadblocks of high learning curves.  By developing a system utilizing widely adopted, well documented, and cross-sector supported standards and solutions the DAP makes it much more likely that new audiences and organizations can and will leverage Folger assets.
* '''International partnerships''' -- As the Folger seeks to develop strong partnerships with similar organizations across the globe it needed a non-proprietary mechanism for connecting with organizations that had chosen different technical infrastructures for managing their own internal assets.  In many ways this is the Folger’s own internal use case for audience expansion, and the partnerships will allow the Folger to develop the DAP with real world experience in sharing, connecting, and juxtaposing their assets with other organizations’ holdings.
* '''International partnerships''' -- As the Folger seeks to develop strong partnerships with similar organizations across the globe it needed a non-proprietary mechanism for connecting with organizations that had chosen different technical infrastructures for managing their own internal assets.  In many ways this is the Folger’s own internal use case for audience expansion, and the partnerships will allow the Folger to develop the DAP with real world experience in sharing, connecting, and juxtaposing their assets with other organizations’ holdings.
* '''Digital Acquisition and Preservation''' -- While many of the original assets the Folger collected were very well understood and the universe of types of content was very static (as it’s rare that a new kind of 1500s era manuscript is discovered) the digital world is in flux and new content types are being defined yearly, and some recent content types are no longer being developed.  In addition the Folger itself is creating new kinds of digital assets out of their physical holdings such as the Internet Shakespeare Editions, Folger Digital Texts archive, and research products such as Folger’s research into female owners of early modern books.” 
* '''Digital Acquisition and Basic Preservation''' -- While many of the original assets the Folger collected were well understood, and the universe of types of content was fairly static (as it is rare that a new kind of 16th-century manuscript is discovered), the digital world is in flux and new content types come into being regularly. Some recent content types are already no longer being developed.  In addition, the Folger is creating new kinds of digital assets out of the physical holdings, such as the Folger Digital Texts archive and surveys of female owners of early modern books.
 


== Major platform components ==
== Major platform components ==
Line 31: Line 31:
System allows individual content items to be flagged as “published” or not, allowing you to keep some data in the system as private / draft content.
System allows individual content items to be flagged as “published” or not, allowing you to keep some data in the system as private / draft content.
=== Schema Microservice ===  
=== Schema Microservice ===  
Allows developers to easily see what content types are configured in any particular installation of the DAP and to retrieve the JSON validation schema that is used to test each content type during import.
Allows developers to easily see what content types are configured in any particular installation of Miranda and to retrieve the JSON validation schema that is used to test each content type during import.
=== Sample Web Client ===  
=== Web Client ===  
Demonstrates how to build a web client that leverages both elastic search and our graphQL api.
A web client that leverages both elastic search and our GraphQL API.
The client also demonstrates how various system integrations can happen in this client middleware.  (For instance using 3rd party viewers to let web users browse and navigate visual assets in the client.)
The client also demonstrates how various system integrations can happen in this client middleware.  (For instance using third-party viewers to let web users browse and navigate visual assets in the client.)
 
=== Repository (AWS) ===
Our cloud-based storage system


=== WordPress Plugin ===
Employing the IIIF Universal Viewer, the plugin allows users to insert Miranda items directly with a shortcode (using the alphanumeric Miranda ID string), which displays the digital object and some top-line metadata. See the plugin in action on the [https://stories.folger.edu/rumors-of-royalty/road-to-revolution/ ''Rumors of Royalty''] Stories site.


== What software and systems is the DAP built with? ==
== What software and systems is the DAP built with? ==
Line 45: Line 50:
* ElasticSearch - https://www.elastic.co/guide/index.html
* ElasticSearch - https://www.elastic.co/guide/index.html
* Pattern Lab - http://patternlab.io
* Pattern Lab - http://patternlab.io
* Amazon Web Services - https://aws.amazon.com
* Cantaloupe - https://medusa-project.github.io/cantaloupe/

Revision as of 15:41, 31 August 2020

What is Miranda?

Miranda is a digital asset platform, a foundation for the Folger's digital collections and one upon which further applications, publications, and initiatives can be built. Miranda's goal is to improve discovery and access in libraries, from making it easier for all users to explore what the Folger has to offer, to improving tools for staff to provide context and information, to increasing efficiency and cost-effectiveness in our digital storage and hosting.

Miranda is being built to help fill a need in the mid-sized cultural institution space that is currently not well served by existing open or closed source solutions. As "born digital" or "digital native" types of content such as databases, blog posts, mobile applications and the like become more commonplace, the existing systems that libraries and cultural institutions have relied on are either too limited, too expensive, or too inflexible to fit our institution's particular needs.

Miranda is an open source software system (licensed under the GNU General Public License v3.0) that allows for the storage of arbitrary binary files and assets and their associated metadata and inter-asset relationships. It has been designed to allow for flexible definition of the kinds of assets it stores, flexible binary file asset storage, enterprise class search, and is built from the ground up to leverage APIs. The DAP has an import component for rapidly ingesting data, an indexing service to manage what metadata is made available for public search, a GraphQL API for asset consumption, and a microservice to publicize configured asset types and their validation schemas.

This project is made possible through the generous support of the Andrew W. Mellon Foundation. Miranda's platform has been designed and built by the ParsonsTKO firm in partnership with the Folger; the WordPress multisite installation has been built by Sites by Coop.

For an overview of the platform, the development process, and the project's ethos for cultural insitutions who may be interested in replicating or modeling the platform for their collections, see the Miranda white paper.

How Miranda helps the Folger:

  • Enhanced remote access -- Not everyone can physically travel to the Folger in Washington, DC. Existing tools were designed to facilitate discovery of assets but not as helpful in the consumption of these assets.
  • Audience expansion -- The Folger currently serves an audience of over 2 million people annually, online and in person. However, many audiences could not be easily targeted with library- or institution-specific data formats, data exchange standards, and other types of technical roadblocks of high learning curves. By developing a system utilizing widely adopted, well-documented standards and solutions, the DAP makes it much more likely that new audiences and organizations can and will leverage Folger assets.
  • International partnerships -- As the Folger seeks to develop strong partnerships with similar organizations across the globe it needed a non-proprietary mechanism for connecting with organizations that had chosen different technical infrastructures for managing their own internal assets. In many ways this is the Folger’s own internal use case for audience expansion, and the partnerships will allow the Folger to develop the DAP with real world experience in sharing, connecting, and juxtaposing their assets with other organizations’ holdings.
  • Digital Acquisition and Basic Preservation -- While many of the original assets the Folger collected were well understood, and the universe of types of content was fairly static (as it is rare that a new kind of 16th-century manuscript is discovered), the digital world is in flux and new content types come into being regularly. Some recent content types are already no longer being developed. In addition, the Folger is creating new kinds of digital assets out of the physical holdings, such as the Folger Digital Texts archive and surveys of female owners of early modern books.

Major platform components

Content Importer

Imports JSON structured content

Content Validator

Validates imported JSON by matching it against an available configured schema. Allows for a hierarchy of validation schemas to be defined with fallbacks.

Content Search

System allows individual content items to be flagged as “searchable” or not, allowing you to keep internal metadata in the system but not junk up your search results with it.

GraphQL API

System allows individual content items to be flagged as “published” or not, allowing you to keep some data in the system as private / draft content.

Schema Microservice

Allows developers to easily see what content types are configured in any particular installation of Miranda and to retrieve the JSON validation schema that is used to test each content type during import.

Web Client

A web client that leverages both elastic search and our GraphQL API. The client also demonstrates how various system integrations can happen in this client middleware. (For instance using third-party viewers to let web users browse and navigate visual assets in the client.)

Repository (AWS)

Our cloud-based storage system

WordPress Plugin

Employing the IIIF Universal Viewer, the plugin allows users to insert Miranda items directly with a shortcode (using the alphanumeric Miranda ID string), which displays the digital object and some top-line metadata. See the plugin in action on the Rumors of Royalty Stories site.

What software and systems is the DAP built with?