Fast Access to Video Information
National Science Foundation Grant No. IRI-9502702
Principal Investigator: Thomas D.C. Little
The use of information retrieval mechanisms (e.g., Mosaic, Gopher) that allow the exchange and display of audio, stills, and video is rapidly becoming a major consumer of network bandwidth and storage. Without filtering mechanisms or hierarchical coding schemes one must process a great deal of data to yield sought information. To support future interactive information delivery services such as entertainment, distance learning, home health care, and telecommuting, there is a need to provide efficient means for locating, accessing, and delivering enormous amounts of video content. In this National Science Foundation-sponsored work, we seek to enable fast access during browsing and retrieval of this video content.
The research addresses an integrated view of multiple technologies necessary to support rapid and high availability access to information. We have focused on the specific areas involving (1) domain-specific information models for video content and capture for hierarchical coding and scalable delivery, (2) video query interfaces tailored to the information models, and (3) mapping of information content to developed delivery protocols for network bandwidth management. The end results are techniques and proof of concept for rapid and efficient delivery of video information while maintaining the ability to scale to many users.
Results were favorable in attaining the proposed research objectives. Particularly interesting results include the development of domain-specific information models for video content using metadata and ontologies; the development of an SGML-based annotation system that supports the data models, capable of GUI reconfiguration based on the captured DTD; the development of video composition techniques for information retrieval and personalization of video content; the development of evaluation metrics for quantifying video composition for news delivery applications; and the development of a proof-of-concept prototype news composition system.Project Participants
In addition to the PI, seven students students received financial support under this grant. These include:
In addition, the Vane project was implemented in part by two exchange students under the supervision of the PI. J. Martin, a senior computer engineering student has also been involved in research as a volunteer. He has contributed significantly to the collection and digitization of analog video for the distance learning and customized-news-service applications. These three students received no financial support under this grant.Other Organizations Involved with the Grant
Two students from the University of Padua were active in investigating topics related to the grant.Activities and Findings
The major research activities, findings, training, and outreach activities of the project are described below.Major Research Activities
A great deal of the research conducted focused on synthesis. As such, the research findings consist of new models and techniques to achieve rapid and personalized video delivery. Major results of this research are:
Graduate student participants have gained a variety of independent research skills while undertaking literature research, mathematical analysis, simulation, and experimentation as part of the project. In addition, students have been expected to produce regular project reports and oral presentations that ultimately have been refined into conference and journal publications. This experience has proven to be invaluable in developing the students into independent investigators.Educational and Outreach Activities
The research sponsored under this grant is promoted via the Web and via various open-house-type activities on campus, including support of Pathways, a program designed to encourage young women to pursue careers in science and engineering.Products
The project has produced a variety of reports, publications, and prototypes, described below.Publications
In addition to publication, the majority of the aforementioned papers appear as technical reports accessible from the project Web address (http://hulk.bu.edu/projects/fap/summary.html).Other Products
A variety of software was developed as proof-of-concepts under the grant. As such, its value has been in the demonstration of ideas rather than as deployable implementations. These ideas are described in detail in the listed publications. As appropriate, developed software is available via the aforementioned Web address.
Canvass, or Customized Access to News Video Archive Storage System, is a proof-of concept prototype video composition system that allows users to retrieve automatically-composed video stories based on input selection criteria. Composition is achieved at the shot level. In essence, users are provided with a set of news stories that are composed, on-the-fly, from available news content, considering their individual needs. We envision this concept to be applicable to future personalized video delivery systems as video storage and delivery systems become more ubiquitous. The Canvass prototype software demonstration system is no longer accessible from the project web site; however, it is described in MCL Technical Report TR-08-25-1999 [pdf] and the source code canvass-src.tar.gz for this application is available as well.Contributions
The project contributed to the multimedia computing domain as well as to other core scientific and engineering disciplines. These, and other contributions educational or societal goals are described below.Contributions to the Discipline
In the past few years there has been an unusual amount of activity in the commercial sector in the multimedia area, especially related to Web technology. This can be interpreted either as competition to the project or as a demonstration of increased significance of the work. Although we have produced prototypes to demonstrate proof of concepts, we view the major work as a conceptual substrate in the form of mathematics, techniques, and algorithms for the development of a variety of tools and applications that will be more robust than our prototypes.
The overall work is largely interdisciplinary, major contributions to various disciplines include (see ``Findings'' for additional details): (1) the development of domain-specific information models for video content using metadata and ontologies, (2) the development of an SGML-based annotation system that supports the data models, capable of GUI reconfiguration based on the captured DTD, (3) the development of video composition techniques for information retrieval and personalization ov video content, (4) the development of evaluation metrics for quantifying video composition for news delivery applications, and (5) the development of a proof-of-concept prototype news composition system.Contributions to other Disciplines
Our work under this grant is inherently multidisciplinary, affecting fields of computer science, engineering, communications, and media production. It is also expected to influence the development of future commercial media production systems. Major interdisciplinary contributions are largely the integration of diverse technologies to yield a functional system that embraces text analysis, database/information system modeling, data storage and delivery, video (media) production, network media delivery, and the use of the Web.Contributions to the Development of Human Resources
Primary contributions are the development of new researchers in engineering through course development, independent graduate student research, and the support of outreach programs.
Teaching activities include the development of courses to reflect the changing needs of computer engineers and the use, evaluation, and promotion of computer-based instructional tools. A hands-on course in multimedia computing was developed proposed that addresses recent advances in computing technology.
In the Spring of 1996, thanks to the recent acquisition of new computer equipment the department for our Software Engineering Lab, the PI introduced a laboratory component consisting of seven two-week laboratory design/experimental elements. These labs were combined with existing student-led seminars and instructor-led lectures. Ultimately these materials have been be polished for use at the mixed graduate and undergraduate level. In the spring of 1997, these materials were augmented with a complete on-line (web-based) environment comprised of lab materials, lectures, and assignments. In subsequent years the material has been refined and refurbished. Although experimentation with video-based instruction was performed in this forum, it was not adopted due to its distraction from the primary educational goals.
The research activities supported by this grant will continue to serve as a substrate for additional research activities in the lab and in undergraduate and graduate teaching. In particular, spin-off projects are encouraged in the course SC748, multimedia computer design, towards related topic areas.Contributions to Physical, Institutional, and Information Resources for Science and Technology
The work under this grant is intended to improve access to information in the video medium. We hope that through promotion and dissemination of our techniques we will ultimately contribute to the development of robust tools and products to achieve this goal in the greater scientific community. Our contributions to date potentially enable a new class of video-based applications that require high-bandwith network delivery. These are anticipated within five years.Contributions to Public Welfare beyond Science and Engineering
Our work is intended to improve the ability to edit and deliver personalized video information content. We expect that our results will lead to more efficient and user-tolerable systems that may see widespread deployment within the next few years, much in the same manner that we have seen an explosion in text-based personalization systems on the Internet.Summary
This research addresses an integrated view of multiple technologies necessary to support rapid and high availability access to video information. Major results include: (1) the development of domain-specific information models for video content using metadata and ontologies, (2) the development of an SGML-based annotation system that supports the data models, capable of GUI reconfiguration based on the captured DTD, (3) the development of video composition techniques for information retrieval and personalization ov video content, (4) the development of evaluation metrics for quantifying video composition for news delivery applications, and (5) the development of a proof-of-concept prototype news composition system.