If the history of art from the Renaissance on may be characterized as a movement from literal pictographic representation to increasingly arcane abstraction, the history of science has more or less followed the opposite trajectory. In the 18th century, the great French mathematician Joseph Louis Lagrange boasted that his monumental opus on calculus and mechanics contained not a single image. Dense with equations, the book represented the ne plus ultra of an ethos that valorized symbolic abstraction at the expense of everything else. Pictures represented, but only equations illuminated the inner laws guiding nature in her course.
For the next 200 years, Lagrange’s view prevailed, particularly in physics and mathematics, where pictures have been seen as weak, woolly substitutes for the crisp rigor of formulas. But at a conference recently at the Getty Center, co-hosted by MIT, this traditional disdain of images was challenged by a formidable group of scientists, who demonstrated in a lavish array of presentations the insights images are beginning to bring to the world of science. Called Image and Meaning 2 (following up on a previous event at MIT in 2001), the gathering celebrated the emerging power of optical and, especially, visual computational techniques in fields as diverse as astrophysics, materials science, complexity theory and molecular biology. This plethora of pictures has been made possible by advances in imaging technologies such as holography, and by vast improvements in the speed and power of computer hardware and software.
In this extravaganza of scientific visualization, a simulated tornado swirled into being while a simulated galaxy spawned infant stars. A drosophila fruit fly embryo, spiked with fluorescent proteins, developed out of a fertilized egg, its genetic activity lighting up the nascent form in a symphony of glowing green. A new technique called “rainbow holographic imaging” revealed the inner depths of a machine part, and a hypnotic video of bubbles clustering into patterns pointed to nature’s mysterious powers of self-organization; these were further illuminated in a demonstration of self-assembling molecules. It was eye candy, to be sure. At the same time, each of these images shed new light onto the underlying science, revealing properties and processes that had hitherto been obscured.
Held over three days, the conference was the brainchild of Felice Frankel, a researcher in scientific photography at MIT. Frankel’s voluptuous imagery of minute and microscopic phenomena, from yeast cells to microchips and nanocrystals, has been gracing the covers of scientific journals for the past decade. She is single-handedly spearheading a campaign to rehabilitate the image in the hierarchy of scientific thinking. “You have to know what you are talking about in order to make pictures of it,” Frankel said, encapsulating her belief that scientific imagery requires knowledge to produce and in turn produces new knowledge.
Frankel’s thesis was epitomized by the National Center for Supercomputing Applications’ (NCSA) stunning animation that simulated the formation of a tornado. As the simulation unfolds, strawlike strands of pressurized air known as stream tubes wind themselves into a vortex and spiral upward, narrowing into the canonical, deadly funnellike form. But the simulation, based on real tornado data, also reveals a phenomenon of which scientists had been ignorant: Halfway through the animation, a small “satellite” tornado breaks away from the main bulk of air and sidles off on its own trajectory. Alerted by this virtual windstorm, the NCSA’s Donna Cox noted, scientists went looking in the real world and discovered that, indeed, actual tornadoes sometimes produce miniature offshoots.
Michael Berry, a physicist at Britain’s prestigious Royal Institution, cited another example of a discovery made through images. Berry showed a rainbow-hued simulation of two light waves passing through a crystal, from which his team discerned a subtle and unexpected anomaly in how light travels in these materials. “It’s pretty arcane,” Berry confessed, “but it is the beating heart of crystal optics. We saw it first in the images, and then we understood it. The moral of the story is that good visual representations lead to better understanding.”
Simulations of light passing
through crystals in different
directions, from Britain’s
Biology is an area in which pictures are increasingly indispensable. Scientists at UC Berkeley are undertaking a massive project to map the action of 1,000 genes during the embryonic development of the fruit fly. What genes are acting at each stage and how much is each one active at any stage? The sheer volume of the information here is daunting. Angela De Pace, a Berkeley biologist who is part of the Drosophila Transcription Network Project and who helped to organize the IM2 conference, marveled along with the audience at the team’s mesmerizing film of a developing fly embryo, its physiology winking with glowing points of green light as various genes turned on and off. Frankly admitting our ignorance in the face of such manifest complexity, De Pace declared that we need completely different ways of thinking about this data.
One question heard throughout the conference was whether we could have a new kind of visual language that would enable scientists to represent and analyze the new kinds of data they are generating. Much of these data map processes rather than structures — genetic and macromolecular processes inside living tissues, the processes in the formation of stars and the processes taking place in our atmosphere that lead to global warming. One of the most arcane yet fascinating sessions was devoted to the topic of how to represent in pictorial form multidimensional sets of data. How, for example, do we begin to make sense of a 17-dimensional “cloud” of data points? Though the end goal may be an equation, there are few tools more astute at picking out overall patterns than the human eye; hence visualizing data in graphical form can quickly reveal relationships that would be obscured by the numbers alone. Even if one does not know at first what the patterns mean, at least one has discovered that they are there — and exploration can begin.
Pat Hanrahan, a computer scientist at Stanford who specializes in graphic representations of complex systems, championed the idea of a “universal visual language.” Though much of Hanrahan’s career has been spent perfecting methods for generating ever more realistic simulations of skin and hair and faces, and translucent materials such as alabaster, his research is increasingly focused on visualizations of the processes inherent in physical phenomena. Hanrahan sees computer graphics as “a powerful tool for scientific problem solving.” Speaking by phone from Palo Alto, he noted that “science is filled with these other levels of abstraction that cannot be captured with realistic pictures.” By “working out how to abstract the essence of a system and transpose it into graphical form,” Hanrahan believes that science is poised for a new era of profound discoveries.
In science as in art, images may be literal representations generated through the now-vast variety of photographic techniques, from X-rays and gamma rays to electron microscopy, magnetic resonance imaging (MRI) and positron emission tomography (PET scanning). At IM2, both genres were celebrated as new sources of scientific insight. As Frankel noted, “The very process of thinking about what you want to represent helps you to understand what you are trying to represent.” The goal here — so splendidly realized — was to re-imagine the world.