INTRODUCTION

As part of a five-year media asset management (MAM) initiative, Scripps Networks launched an acquisition process for an enterprise DAM application in November 2005. Tied intimately to this five-year initiative, was the implementation of service-oriented architecture (SOA) to enable application interoperability and process automation. Scripps Networks needed to acquire a DAM application that offered both a strong (but not exclusive) video management capability and played well in a services environment. This presented a unique challenge, both from defining and specifying the needs of our SOA, and in presenting our requirements in a form that would allow us to objectively score the offerings of various vendors.

CREATING THE DAM SOA REQUEST FOR PROPOSAL (RFP)

Prior to creation of the RFP

In the case of Scripps Networks, the issuing of a DAM SOA request for proposal (RFP) was not an isolated event, but rather the culmination of a series of events going back over a year. Scripps Networks had started this process in 2004 by undergoing an extensive re-evaluation of its strategic needs for MAM. The re-evaluation surfaced the high-level business needs, as well as creating a roadmap of enterprise capabilities. Fundamental to the expression of those capabilities, was the use of SOA. The strategic re-evaluation produced a number of artifacts that were critical to the DAM RFP process. Those findings included:

  • business and strategic drivers;

  • preliminary enterprise reference architecture;

  • current state of MAM architecture;

  • future state of MAM architecture;

  • asset lifecycle.

Generating this information required a great deal of focus by the organization and resulted in executive-level buy-in to the MAM initiative and projected costs. While ROI was (and is) important, the business saw the potential efficiencies and the ability to scale business operations as equally important.

The other benefit to the groundwork performed prior to issuing the RFP was a depth of shared understanding about the expected role of DAM within the overall architecture. Business users and information technology (IT) were able to focus on satisfying business needs and not on just generating functional requirements.

Expressing SOA specifics

So, how would we take this wealth of information about the business needs and translate it into a format that would elicit the vendor's view of how its product fits Scripps Networks’ SOA? We approached this challenge by combining traditional functional/nonfunctional requirements with business-driven scenarios. We used four different formats to express those business needs and processes within the SOA framework:

  • global use cases;

  • user community definition;

  • production deployment scenario;

  • preliminary enterprise reference architecture.

Of the four formats, only the global use cases have any basis in unified modeling language (UML) or an equivalent modeling language. The intent was, however, not to drive code design, but rather to produce concrete information from the vendors on their approach to SOA.

We expected the business-oriented information to provoke a more verbose reply; vendors would be unlikely to reply with UML that could be analyzed for accuracy and completeness. For the technical, operational and business requirements, we, however, provided a self-tabulating Microsoft® Excel© worksheet that allowed the vendors to score their offering against our criteria.

Use cases

Figure 1 provides a sample use case presented in the DAM SOA RFP.

Figure 1
figure 1

Sample use case

Full size image

Multiple use cases were provided as part of the RFP, spanning workflows for broadband and broadcast video content and the range of the user community interaction with the DAM application.

User community

The second approach Scripps took in expressing business needs was in defining the user community and the workflows to be executed via the DAM. Figure 2 represents the user community within Scripps Networks for the DAM SOA. The illustration itself provides modest value, but with each defined user community we wrote a paragraph describing their expected interaction with DAM SOA. A sample of this interaction is provided in immediately below.

Figure 2
figure 2

User community for DAM

Full size image

Business and legal affairs

Business and legal affairs requires access to media content to support the following major objectives: (1) validating the correlation between third-party media content utilized and compensation arrangements with talent including any applicable royalties, residuals, talent payments/fees, etc and (2) ensuring that produced content meets the guidelines for suitability (also known as a review of suitability of content standards and practices).

Production deployment scenario

While not directly an artifact for SOA analysis, the DAM SOA RFP defined a production scenario based upon nonfunctional requirements (eg scalability, availability, performance), so that vendors could respond with a proposed architecture, instead of just providing us with a “parts list.” Vendors were asked to respond with a software and hardware deployment diagram showing all licensed parts provided in their estimate.

Enterprise reference architecture

A necessary element of SOA development is a reference architecture that provides a shared vision of the envisioned services and how they are organized. As part of the DAM SOA RFP, Scripps Networks provided the high-level reference architecture shown in Figure 3. This diagram drives home the point that Scripps Networks requires the DAM service interface to function as part of a larger SOA framework, including distributed metadata repositories, federated search, automated workflows and interoperability across disparate media management systems. The shaded boxes within each meta-category (ie metadata management) represent those services areas most connected to DAM SOA.

Figure 3
figure 3

Scripps Networks’ SOA reference architecture

Full size image

In addition, Scripps Networks expressly defined a hierarchy of critical features relating to how the DAM would be used. In prioritized order from most to least important, the following features were identified:

  • services access to key DAM functionality;

  • support for complex media workflows in a SOA orchestration context;

  • portal integration;

  • Java interface;

  • native UI.

This order reflects the reverse of typical DAM application access via a custom UI and limited programmatic control. It also defines a clear need for standards-based services capable of orchestration into multi-stage media workflows.

Functional, technical, operational and business requirements

In addition to the SOA-oriented information described above, Scripps also provided a very complete set of traditional functional requirements. We knew that vendors would gravitate toward addressing and responding to these more easily than the SOA components. The drawback with requirements listed in this fashion, is that it is not easy to impart business context to each requirement. Requirements can, however, be useful to gather information about the boundaries of the core application and what the vendor really understands about the feature. For instance, we asked extensive questions about metadata types and searching — knowing fully well that enterprise search would be outside the scope of the DAM SOA. What we really wanted to know was how well their product would support federating that information in an external search engine.

DAM SOA RFP RESULTS

Once all the RFP content had been assembled, Scripps Networks solicited 16 vendors to respond to the DAM SOA RFP. These vendors ranged from “pure play” DAM vendors to SOA technology platform vendors. Out of those 16 solicitations, we received ten responses. It is interesting to note that (and very telling) after the RFP was issued, a number of vendors formed partnerships in order to reply to the RFP. Pure play DAM vendors looked for SOA providers and vice versa. As a result of new partnerships that were formed, some vendors lost their traditional partners and were unable to respond.

As mentioned previously, we included an Excel worksheet that contained all objective criteria in a scoring template. Vendors were able to score themselves against a given criteria and provide additional comments as needed. What was not exposed to vendors was a weighting factor for each criteria and section. This gave Scripps Networks the ability to weight the requirements sections that were most critical, without allowing the vendor to be able to “game” the scoring. The Excel spreadsheet summarized the scores by section and provided an overall score for the response.

While all this sounds highly useful for providing an objective view of responses, in practice, we still had a tremendous amount of work ahead of us to adequately score the responses. It quickly became clear that vendors are not the most objective people in the world and we needed to read the responses and revise the scores to reflect the responses more accurately. So, while the scorecards provided a nice summary, we found that it would have been just as effective to score the responses ourselves. The vendor scores did prove to be a constant source of humor during the review process.

There are several salient points worth mentioning in looking across all the vendor responses. The projected five-year costs had a very wide spread from the low hundreds of thousands to multiple millions. This represented a significant spread for vendors reading the same set of requirements.

When the weighted scoring of the responses was tabulated, the highest score (on a relative basis of 0–100 per cent adherence to requirements) was 87 per cent and the lowest was 47 per cent. Increasing cost generally correlated with the higher weighted score but there were exceptions. The selected vendor did not score the highest weighted score and did not have the lowest price.

As mentioned above, we used several unconventional means to express business context and to define Scripps Networks’ SOA to the vendors. The reaction to those components of the DAM SOA RFP was interesting as well.

Use-case review

The response to our SOA use cases was tepid at best. While use cases may have been a great vehicle by which Scripps Networks could express business context to the DAM vendors, the responses mostly just assured us of their ability to meet our needs. There were little or no specifics on how they would meet our requirements, which specific DAM SOA service calls would be used, or if any development was required. In short, we were not able to get the vendors to respond in a meaningful way to the SOA use cases.

Service interface review

As part of the vendor's response, Scripps Networks requested documentation on all service interfaces and programmatic application programming interfaces (APIs) provided as part of the product. This actually proved to be more useful in evaluating the vendor's level of maturity with regard to SOA. Vendors varied a great deal in their response to this request — from extensive documentation for hundreds of API calls to none at all. Where documentation was provided, Scripps Networks was able to evaluate the services interfaces for complexity, granularity, and to assure a good fit to our use cases.

Final round evaluations

Through several rounds of review and scoring, Scripps Networks narrowed the field down to two DAM SOA candidates. Each of these candidates was then asked to present its solution to an internal review committee. A review meeting was scheduled and the vendors were asked to bring their management team and the personnel with whom Scripps Networks would be working in the event of a purchase. The intent of this meeting was interaction with the core management team and to get a feel for them as individuals. The meeting also gave the vendors a final chance to offer any additional financial incentives to Scripps Networks.

The scoring before and after the next to final round was extremely close. Both vendors had some level of proprietary code base, but had documented service interfaces that were in production usage and/or at an advanced stage of development. In the final analysis, Venaca™ was selected with the deciding factors for the selection comprised of outstanding customer references in Scripps Networks market space, direct experience with broadcast video assets and a well-designed set of services interfaces which were compatible with the needs of Scripps DAM SOA reference architecture.

But the acquisition process was not yet complete. Now we would take the final vendor through a detailed pilot. This was important because it is far easier to author a compelling RFP response than it is to deliver high-quality features, reliability, scalability and performance in a test pilot with a demanding production environment. And so, this DAM SOA pilot was focused on defining the parameters of a pilot project to perform important validation work prior to a financial and production rollout commitment with any DAM SOA supplier on the shortlist.

DAM SOA PILOT

Pilot objectives

Overall, the goal of the pilot was to evaluate the system across a comprehensive set of criteria to support a key decision — should Scripps Networks move forward with a DAM SOA solution selected in the RFP process? To support this objective, a checklist/scorecard approach was used, whereby system characteristics were qualitatively and quantitatively evaluated relative to our overall goals for a new DAM system. If the pilot system made the grade, then Scripps Networks would proceed to development and deployment of a production system. Conversely, if the pilot system did not meet the criteria, then an alternate system might go into a sequential second pilot, or the initial system might need to be tuned or reconfigured in some manner in order for it to be considered acceptable for production.

The concept of establishing gates and milestones relative to proceeding with a new DAM system was critical, based upon the legacy of challenges with prior DAM systems, as well as the business-critical nature anticipated for the new solution. And so, upon completion of the various tasks and tests outlined in this document, the checklist/scorecard had to be completed to support key decisions required for a potential “green light” into production mode. This mandated development and evaluation of a comprehensive set of metrics for the DAM SOA pilot. These metrics are described in the following section.

Pilot metrics

In addition to a technical validation of the selected DAM SOA solution in the pilot phase, a more accurate understanding emerged for the definition of the production architecture. Accuracy of the production system software, hardware and network requirements supported completion of financial costing and procurement needs for the production rollout. Likewise, a more detailed Pilot Project Plan, which would serve as our contractual framework with suppliers and partners for the pilot, was the logical next step upon completion of the DAM SOA pilot project.

Key topics and metrics reviewed in the Pilot Project included the following:

  • Use-case validation: Four DAM SOA RFP use cases were selected for focused deployment in the pilot project, with those use cases prioritized by Scripps Networks. These use cases were characterized as “pass” or “fail” based on a specific set of evaluations required for each real world use case.

  • User group participation: One or more user groups and users with specific roles within those groups were targeted for participation in the pilot project.

  • Ease of installation and configuration: Metrics were defined to measure how difficult or how easy the basic system was to install and configure to achieve operational status.

  • Functional depth and breadth: The DAM SOA RFP included more than 150 specific product requirements that were deemed important for the DAM SOA platform. The pilot project was geared to “spot check” as many of the high-priority features as possible, and correlated stated abilities in the RFP response with actual capability of the product in pilot operation. Any significant deviations of capability found in the pilot relative to the RFP response were investigated and remedied prior to production rollout.

  • Performance, reliability and scalability: The performance, reliability and scalability of the platform are critical determinants of success for use of the DAM SOA platform in operation. The pilot was defined to measure these metrics and to do so in a way that validated the use of the proposed DAM SOA system prior to production usage. Simulated load testing would also be a key aspect of validating performance, reliability and scalability characteristics.

  • Ease of integration and interoperability: Ease of integration and interoperability with broadcast and broadband infrastructure at Scripps Networks was selectively evaluated within the context of the pilot.

  • Develop production cost model: The pilot planning and rollout processes established a cost model for the full implementation of the DAM. This included hardware, software licenses, customization and resources.

Each of these areas is examined in more detail below.

Use cases

Using the four use cases introduced in the DAM SOA RFP, plans were developed to specify how the use cases would be tested and validated within the context of the pilot, with additional detail provided on workflow in order to support pilot deployment.

The areas of focus for use-case validation, in order of priority, included the following, where a simplified version of each use case was pursued in order to meet the timeline, budget and resource requirements of the DAM SOA pilot:

  • Delivery of media proxies: Access to digital asset metadata and proxies from Scripps Networks applications via programmatic single sign-on access.

  • Digital media delivery on-demand: Flexible and high performance transcoding of source media into targeted formats with distribution to various locations (ie FTP, HTTP, etc).

  • Distributed video production: Ability to support rough-cut edits at the desktop, which can be exported for final edits in a video editing suite/application; optional enhancements to include testing of support for distributed proxies and media.

  • Digital media proxy/asset ingest: Acceptance and automatic registration of video proxies and/or assets from other applications into the DAM.

User participation

Multiple user groups and users with specific roles within those groups were targeted for participation in the pilot project. Pilot project users were included from one or more of the following user groups: Broadcast and Broadband Programming, Sales and Marketing, Operations, Creative Services, Post Production and Business and Legal Affairs. User groups with frequent access needs were targeted for inclusion in the pilot.

Selected objectives for the pilot were of a more technical nature and included evaluations of load and stress testing, confirmation of the solution's ability to provide stated functional requirements, test application development for DAM SOA interfaces, etc. The inclusion of business personnel and end users in the pilot had to be approached carefully, given the workloads and responsibilities of key staff members.

Objectives for user group participation were summarized as follows:

  • Broadcast programming: Broadcast programming staff members evaluated the utility of DAM SOA to access metadata and proxies in core Scripps Networks applications as well as rough-cut video edit functions at the desktop.

  • Broadband programming: Broadband programming staff members evaluated the utility of DAM SOA access to metadata and proxies in core Scripps Networks applications, transcoding capabilities of the platform to flip media to VOD and broadband syndication partners and rough-cut video edit functions at the desktop.

  • Broadcast operations: Operations staff evaluated the efficiency of transcoding source media into multiple formats, which would then be used to deliver content to broadband and VOD partners, and the performance of content distribution to multiple targets.

  • Creative services: Creative staff was asked to evaluate the efficiency of searching and retrieving assets from the DAM for asset repurposing efforts.

  • Scripps productions: Scripps productions staff was asked to evaluate the efficiency of searching and retrieving assets from the DAM for asset repurposing efforts.

  • Tape library/media center: Staff members who manage the Scripps Networks tape library and media center were asked to evaluate the potential for using the DAM solution as an alternative to the existing tape library system. This approach could prove to be valuable in support of a centralized repository that contains both physical and digital assets. A single repository of both classes of assets could provide end-user synergies along with cost effectiveness.

  • Post production: Post production staff was asked to evaluate the efficiency of searching and retrieving assets from the DAM for the purpose of working on post-production projects.

  • Software development: The Scripps Networks software development team was asked to evaluate the ease of use, ease of integration, documentation, reliability and performance associated with programmatic control of the DAM SOA platform. Test applications were to be developed to exercise the DAM SOA interfaces — even small “applets” in order to put the DAM SOA solution through its paces.

  • Quality assurance: The Scripps Networks quality assurance team was asked to evaluate the load and stress testing for the DAM SOA platform; this approach included scripts which automated manual end user DAM functions.

  • System administration: The Scripps Networks system administration team was asked to evaluate the ease of use, configurability and overall quality of the DAM SOA administration platform.

Installation and configuration

Metrics were defined to measure how difficult or how easy the solution was to install and to configure to achieve operational status. For example, the following criteria were evaluated:

  • installation elapsed time;

  • installation internal resources;

  • installation external resources;

  • ease of installation;

  • reliability of installation;

  • configuration elapsed time;

  • configuration internal resources;

  • configuration external resources;

  • ease of configuration;

  • reliability of configuration;

  • flexibility of configuration;

  • ease of UI configuration;

  • ease of metadata configuration.

Software development, system administration and quality assurance personnel worked directly with the vendor during installation and setup to evaluate the metrics.

RFP functional requirements

The DAM SOA RFP included close to 300 technical and business requirements, with more than 150 specific product requirements that were deemed important for the DAM SOA platform. A subset of functional requirements was identified for pilot testing, with specific goals outlined to detail specifically how and what to test.

Functional DAM SOA areas such as asset metadata/cataloging, ingest, versioning, collaboration/workflow, transcoding, search, permissions, administration, ordering, reporting, etc. were reviewed with selected requirements identified for testing in the pilot phase. For each area, a test was designed to determine if the DAM system did in fact provide the capability as stated in the RFP response. Any discrepancies were noted using a scoring spreadsheet with notes taken relative to the qualitative aspects of using the stated feature.

PRODUCTION ACQUISITION

Production acquisition required Scripps Networks to evaluate the DAM application under real-life conditions via aggressive load testing. The load-testing process evaluated production hardware requirements in the pilot phase and drove computer processing unit (CPU) licensing models for financial purposes. The objectives of the load testing were as follows:

  • obtain system scalability data from the pilot to support development of projections of production system resources needed;

  • verify pilot capabilities to support 100 concurrent users on baseline hardware.

Mercury™ LoadRunner® was used to script selected DAM transactions available using a web interface. The following transactions were scripted, based upon limited capabilities available through a limited scope web interface:

  • search by browsing hierarchy — 40 per cent of users;

  • search by Keywords — 40 per cent of users;

  • downloading files — 20 per cent of users.

The results of load testing indicated that CPU utilization stayed well below 100 per cent across all tests (this parameter peaked at 35 per cent for the database server and 25 per cent for the application server). Response time for search was acceptable for up to 75 concurrent users (10 seconds or less) on the limited pilot hardware; additional memory and processors and/or new database structure were needed to go beyond 75 concurrent users with acceptable search response times. With initial pilot testing, response time for browse and download was acceptable for a limited number of users; this required additional investigation. Response time varied in a predictable manner for search response time as a function of the number of concurrent users. The transaction error rate scaled reasonably well (less than 1 per cent at 100 users) and was kept within reasonable limits at higher user counts. Good results for search response time, CPU utilization and transaction error rate were offset by issues related to response time for browse and download. These data were analyzed and then used to support further testing and evaluation.

Leveraging the load testing results and data analysis, the production system was planned as a scaled up version of the pilot system adding redundancy at many levels of the architecture such as multiple application and database servers, increasing (quadrupling) the amount of memory in primary servers and replicating servers such as transcoders to support anticipated load. It is probable that hardware load balancers will be needed on the front end of the deployment, so as to distribute incoming user requests across the primary application servers. Overall, the load-testing process was critical to identify bottlenecks and to design an appropriate redundant, scalable and load-balancing production system to handle anticipated loading.

The load testing configuration used during the pilot is shown in Figure 4.

Figure 4
figure 4

Pilot architecture used for load testing of the DAM platform

Full size image

CONCLUSIONS

At the time of writing this paper, the pilot project has been completed and Scripps Networks has installed the Venaca S3 application into production. On a relative basis, the DAM SOA pilot produced good results with some limitations. The ultimate success of the DAM SOA across the enterprise has, however, not yet been established. Scripps Networks has just begun the process of integrating the DAM SOA into its overall architecture and to define new workflows. Given that state, conclusions regarding the acquisition process are limited only to the RFP and pilot project at this time; summary points are provided below as follows:

  • Most, if not all, DAM vendors are not technically positioned to fully participate in a SOA. They simply cannot offer a complete solution without working with partners to augment their core product.

  • The corollary to the point above is that buyers should not be afraid to mix and match vendors to aggregate the functionality required for a DAM SOA deployment.

  • Major gaps exist in the DAM offerings evaluated by Scripps Networks:

    •    Significant limitations surrounding the consumption of external services, especially external security services relating to user permissions.

    •    Many solutions still rely upon proprietary application stacks that have not embraced standards (ie Business Process Execution Language (for Web Services) (BPEL)) to achieve service integration and portalization of composite applications.

    •    Create Read Update Delete (CRUD) operations against assets in the DAM via vendor services often require that the developer have intimate knowledge of the underlying database schema. This is contrary to general SOA practices, where knowledge of the underlying application should be abstracted from the service.

  • Piloting of the application was critical to the success of the acquisition process. Despite our best efforts to drive out issues during the RFP paper-based process, we still uncovered quite a few critical issues during the pilot phase in an actual implementation. We could not have accurately defined and mapped our production deployment or licensing agreement without having executed the pilot.

Procuring a DAM SOA requires the buyer do a great deal more preparation to define the expected role of the DAM application within an enterprise services framework prior to the acquisition. At a minimum, SOA reference architecture, media workflows and business-level use cases for the DAM should be defined. Going further, a DAM SOA pilot should be planned that will include implementation of the most important media workflows in a SOA context along with detailed scoring of key metrics as outlined in this report. DAM SOA is at an early stage of development; however, based on our experience there is an ability to deploy basic solutions which address key use cases and we expect richer SOA support in the coming time period. We hope the DAM community strongly considers the importance of requirements in this area as more and more of the IT world moves towards SOA frameworks to support business needs in an agile and efficient manner.